Even when ionic compounds are strong (e.g. rubies and sapphires), they are neither malleable nor ductile in the slightest, and if sufficient stress is put on them, they will shatter, not bend. Why is that?
Ionic crystals are hard because of tight packing lattices, say, the positive and negative ions are strongly attached among themselves.
So, if mechanical pressure is applied to an ionic crystal then ions of similar charges may be forced to get closer to each other.
Now, by doing so, the electrostatic repulsion can be enough to split or disorient completely the lattice infrastructure. Thus imparting the brittle character.
They’re brittle because it’s very difficult for dislocations to move through the crystal lattice. Dislocations are what mediate plastic deformation in crystals, so the fact that creating and moving a dislocation in an ionic crystal incurs a massive energy penalty means that an ionic crystal will undergo brittle fracture.
Ionic compounds are not unusual in being brittle
Your starting assumption that ionic compounds are frequently brittle is misleading. Many, if not most, solids are brittle, ionic or not.
The reasons why things are brittle have more to do with the bulk structure of the material and less to do with the chemical makeup of the material.
Table salt is an ionic compound and is brittle. But diamond is also brittle despite being a molecular solid where all the carbon-carbon bonds are covalent. But forged iron is strong and far from brittle. Pure copper is soft and malleable and not brittle. Nylon and Kevlar are the opposite of brittle.
Strength is more or less unrelated to being brittle or not and, defined properly, is a measure of the ability to resist deformation. But this is almost unrelated to brittleness. Glass is very strong but, like salt, very brittle which is why dropping your phone on hard surfaces is a bad idea.
Toughness is a better term for the opposite of brittleness. Tough compounds can deform without shattering. Nylon is weak but tough, kevlar is strong but tough, forged iron is also strong and tough. But cast iron is strong but brittle, providing a hint that the overall chemistry isn't everything.
What actually makes for tough compounds is the ability to mitigate external stress in the molecular structure of the material. In many polymers, the bonds in the long polymer chains can rotate and rearrange themselves to relieve that stress. In some metals (forged iron but not cast iron) the crystal structure of the metal contains defects that can move and rearrange to relieve stress concentrations. In contrast glass and table salt can't do that and even small surface scratches concentrate stress and rapidly grow causing the compound to shatter. They lack a molecular mechanism to mitigate the stress concentrations caused by small cracks. This can be partly overcome by more complex treatments of the surface of the compound. "Strong" glass (like the gorilla glass used for mobile phone screens) uses a process that treats the glass to create tension in the surface. That tension minimises the stress concentrations from small surface scratches and makes the resulting glass much stronger (this is sometimes achieved by deliberately adding ions to the surface of the glass).
Toughness is a product of the bulk material not of the bonding type of the molecules or atoms that make it up.
In summary, brittleness is not a property uniquely associated with ionic compounds. Most crystals are brittle. Those compounds that are not brittle are not distinguished by the type of bonding involved but by complex mechanisms that can alleviate the concentration of stress in the bulk compound. So many covalent crystals are brittle not just ionic ones. Some metals are brittle, though many are not.