Instead of trying to use symmetry tables it is possible to calculate the vibrational normal modes directly using the equations of motion, rather as would be done for a double pendulum for example.
It is normally assumed that the vibrational motion of the atoms is harmonic, i.e. Hook's law applies, and we additionally assume that the only force constants are those between adjacent atoms; this is equivalent to assuming that the valence bond model describes the bonding. A molecular orbital model would consider forces constants between one atom and every other atom.
The starting point is to calculate the potential energy and, from this, the forces on the atoms. As carbon dioxide is symmetrical, there is only one force constant, k, but as the masses are different it is found that the mass-weighted force constants are needed in the calculation and these are different for each type of atom.
Stretching vibrations
The normal modes for a simple molecule can easily be sketched and as the molecule is linear there are $3N − 5$ or four modes in total, two are stretches and the other two are the degenerate bending motion in the plane of the page. One where the carbon atom moves down and the two oxygen atoms move up and then vice versa. The other bending normal mode is the similar motion but perpendicular to the plane of the page. Only the stretching modes are considered here.
Placing a vector $s_1,\,s_2,\,s_3$ on each atom and pointing in the same direction and along the long axis of the molecule, the potential energy V is the sum of terms for the stretching of each bond; therefore,
$$V(s_1,s_2,s_3)=\frac{k}{2}(s_1-s_2)^2+\frac{k}{2}(s_2-s_3)^2 $$
These equations are just for the stretching motion, a second calculation is needed with vectors perpendicular to the molecular axis for the bending modes.
The forces are the negative derivatives with respect to the displacements
$$-\frac{dV}{ds_1}=-k(s_1-s_2); \quad -\frac{dV}{ds_2}=k(s_1-s_2)-k(s_2-s_3);\quad -\frac{dV}{ds_3}=k(s_2-s_3)$$
With f as the force, placing these equations into matrix form gives
$$\begin{bmatrix}
f_1\\ f_2\\f_3
\end{bmatrix}
=
\begin{bmatrix}
-k & k & 0 \\
k & -2k & k\\
0 & k & -k
\end{bmatrix}
\begin{bmatrix}
s_1\\ s_2\\s_3
\end{bmatrix}$$
but, because the masses are different, we must change to mass weighted force constants, using the formula $K_{i,j}=k_{i,j}/\sqrt{m_im_j}$ where i and j are the atom indices; for example $K_{1,2} = k\sqrt{ m_Om_C}$, is the mass-weighted force constant between atoms 1 and 2, if $m_O$ is the oxygen mass and $m_C$ that of the carbon. The matrix of force constants becomes
$$\boldsymbol{K} =
\begin{bmatrix}
\displaystyle\frac{-k}{m_O} & \displaystyle\frac{k}{\sqrt{m_Om_C}} & 0 \\
\displaystyle\frac{k}{\sqrt{m_Om_C}} & \displaystyle\frac{-2k}{m_C} & \displaystyle\frac{k}{\sqrt{m_Om_C}}\\
0 & \displaystyle\frac{k}{\sqrt{m_Om_C}} &\displaystyle \frac{-k}{m_O}
\end{bmatrix}$$
which is solved as a secular determinant with eigenvalues $\lambda$,
$$ \lambda_1=-k/m_O; \quad \lambda_2= -k\frac{2m_O+m_C}{m_Om_C}; \quad \lambda_3 = 0 $$
The frequency of each vibration is $\omega^2 = −\lambda$ so the square of the normal mode frequencies are
$$\omega_1^2=k/m_O; \quad \omega_2^2 = k\frac{2m_O+m_C}{m_Om_C} ; \quad \omega_3=0$$
Using the measured stretching vibrational frequencies for $\ce{CO2}$, for example, which are 1337 & 2349 cm$^{-1}$, the force constants are 1680 & 1418 N/m respectively. Note that the frequency $\omega = 2\pi\nu c$ with $\nu$ in cm$^{-1}$.
The normalised eigenvector matrix x is, after some rearranging and with the total mass as $M=2m_O+m_C$
$$\boldsymbol {x}= \frac{1}{\sqrt{2M}}
\begin{bmatrix}
-\sqrt{M} & \sqrt{m_C} & \sqrt{2m_O} \\
0 & -2\sqrt{m_O} & \sqrt{2m_C}\\
\sqrt{M}& \sqrt{m_C} & \sqrt{2m_O}
\end{bmatrix}$$
Notice that the eigenvectors do not depend on the force constants; this matrix is used to produce the geometry that is related to the symmetry of the vibrations, and cannot depend on the value of the force constants. The normal modes depend only on the geometry because the ‘springs’ connecting the atoms can only vibrate in certain patterns governed by the geometry or symmetry of the molecule; the frequency and size of extension depend on the force constants.
The normal mode coordinates are calculated using $\boldsymbol{Q = x^Tq}$ where $\boldsymbol{x}^T$ is the transpose of the eigenvector matrix and q the vector of mass weighted coordinates with $q=s\sqrt{m}$.
$$\begin{bmatrix}
Q_1\\Q_2\\Q_3
\end{bmatrix}
=
\begin{bmatrix}
-\sqrt{M} & 0 & \sqrt{M} \\
\sqrt{m_C} & -2\sqrt{m_O} & \sqrt{m_C}\\
\sqrt{2m_O}& \sqrt{2m_C} & \sqrt{2m_O}
\end{bmatrix}
\begin{bmatrix}
s_1\sqrt{m_O}\\s_2\sqrt{m_C}\\s_3\sqrt{m_O}
\end{bmatrix}$$
and the individual modes are
$$Q_1=\sqrt{m_O/2}(-s_1+s_3); \quad Q_2 = \sqrt{\frac{m_om_C}{2M}}(s_1-2s_2+s_3);\\ \quad Q_3=(m_Os_1+m_Cs_2+m_Os_3)/\sqrt{M}$$
The symmetry of the normal mode can be seen from the displacements s. The last, $Q_3$, is just a translation of the molecule as all the s vectors point in the same direction and has frequency $\omega=0$. $Q_1$ would correspond to a symmetric stretch and therefore $Q_2$ to the assymetric stretch, both O atoms move in one direction and the carbom moves the opposite way to keep the centre of gravity fixed in space.
We can choose any Q to be zero to find the s displacements
(i) Suppose we choose $Q_1 = Q_2 = 0$, then all the atom displacements are the same, $s_1 = s_2 = s_3$ and are $Q_3/\sqrt{M}$. As each atom is moving in the same direction this cannot be a vibrational normal mode as the bonds are neither stretched nor compressed, but represents a translation, and clearly, this corresponds to the zero frequency eigenvalue $\omega_3 = 0$. Note that if $Q_3 = 0$ this must mean that the centre of mass does not change, no translation occurs, and so we expect to produce normal modes with this condition.
(ii) If $Q_1=Q_3=0$ then $\displaystyle s_1=s_3= \frac{m_C}{2m_OM}Q_2$ and $\displaystyle s_2= - \frac{2m_O}{2m_OM}Q_2$ In this case this is the asymmetric stretch with the oxygen atoms moving in the same direction and opposed to that of the carbon, and this is $\omega_2$. The relative motion is $s_1 = s_3 = 0.115Q_2$ to $s_2 = −0.308Q_2$ so the carbon atom moves further than the oxygen atoms do, which is not surprising, because the centre of mass has to be held constant and the C atom has to compensate for the motion of two O atoms.
(iii) Finally, if $Q_2 = Q_3 = 0$ then the equations are solved when $s_2 = 0$ and $s_1 = −s_3$, which is the symmetric stretch $\omega_1$ with displacements $\pm Q_1/\sqrt{2m_O}$ or $0.176Q_1$.
There is a very general method with which to do these types of calculations called the GFG matrix method or some similar name equivalent to this. As this is a matrix method it is easy to calculate equations using symbolic algebra programmes such as SymPy or to do the calculation numerically using for example Python. (Both SymPy & Python are free to use).
The secular determinant to be solved is
$$ | \boldsymbol{GFG} -\omega^2\boldsymbol{I} |=0$$
where I is a unit diagonal matrix, $\omega$ the normal mode frequencies and the matrices are G is a matrix zero everywhere except on the diagonal where it has values $1/\sqrt{m}$, i.e.
$$\boldsymbol{G} = diag \left [ 1/\sqrt{m_i}\right ] $$
and the F matrix is that of the force constants;
$$\boldsymbol{F}
=
\begin{bmatrix}
k_{11} & k_{12} & k_{13} & \cdots\\
k_{21} & k_{22} &\cdots & \cdots \\
\vdots & \vdots & & k_{nn}
\end{bmatrix}$$
and the product is just the mass weighted matrix of force constants
$$\boldsymbol{GFG}
=
\begin{bmatrix}
k_{11}/m_1 & k_{12}/\sqrt{m_1m_2} & k_{13}/\sqrt{m_1m_3} & \cdots\\
k_{21}/\sqrt{m_1m_2} & k_{22}/m_2 &\cdots & \cdots \\
\vdots & \vdots & & k_{nn}/\sqrt{m_n}
\end{bmatrix}$$
The eigenvectors x can be used to produce the normal mode displacements as $\boldsymbol{Q=x^Tq}$ where $\boldsymbol{q=G^{-1}s}$ so that each element is $q_i= s_i\sqrt{m_i}$ and the coordinate displacements via, $\boldsymbol{ s =GxQ}$.
Bending vibrations
In the case of bending vibrations in a molecule with masses $m_{1,2,3}$, such as HCN, the potential is written as
$$V= k_br_1r_2(\delta\theta)^2/2$$
where the bond lengths are $r_{1,2}$ and the bending force constant $k_b=k_\theta r_1r_2$ which keeps its units in N/m. For small angle bends $\displaystyle \delta\theta = \frac{(s_1 - s_2)}{r_1} - \frac{(s_3 - s_2)}{r_2}$ (where vector s is perpendicular to the internuclear axis just as s was along the axis) and the mass weighted K matrix of force constants becomes
$$K =
\begin{bmatrix}
\displaystyle \frac{k_3}{m_1}\left(\frac{r_2}{r_1}\right) & \displaystyle\frac{-k_3}{\sqrt{m_1m_2}}\left(1+\frac{r_2}{r_1}\right) & \displaystyle \frac{k_3}{\sqrt{m_1m_3}} \\
\displaystyle\frac{-k_3}{\sqrt{m_1m_2}}\left(1+\frac{r_2}{r_1}\right) & \displaystyle\frac{k_3}{m_2}(2+\frac{r_2}{r_1}+\frac{r_1}{r_2}) & \displaystyle \frac{-k_3}{\sqrt{m_2m_3}}\left(1+\frac{r_1}{r_2}\right)\\
\displaystyle \frac{k_3}{\sqrt{m_1m_3}} & \displaystyle\frac{-k_3}{\sqrt{m_2m_3}}\left(1+\frac{r_1}{r_2}\right) & \displaystyle\frac{k_3}{m_3}\left(\frac{r_1}{r_2} \right)
\end{bmatrix}$$
which means that the ratio of bond lengths must be known if they are different from 1.
The algebraic eigenvalues are immensely complex so it is necessary to calculate them numerically except for the case of $D_{\infty h}$ point group. In this case the force constants are zero, or $2k_3(2/m_2 +1/m_1)$ where $m_2$ is the central atom.
In the case of $\mathrm{CO}_2$, $m_1 = 16$, $m_2 = 12$ and the bond length $r=0.116$ nm. The only non zero (and doubly degenerate) frequency is $\omega^2 =2k_3(2/m_2 +1/m_1)$ . In $\mathrm{CO}_2$ the bending vibration has a frequency of $667$ cm$^{-1}$ and then $k_3 = 57$ N/m.
(source. The arguments presented here follow closely that in Beddard 'Applying Maths in the Chemical & Biomolecular Sciences, an example based approach' publ OUP, and where there is a more detailed description. Herzberg gives several examples (Vol II, chapter 2) and Wilson, Decius & Cross 'Molecular Vibrations describe the method in great detail.)