Alarms will sound when one of two things happens:
One of your soldiers was spotted by an enemy soldier or a surveillance camera.
A downed enemy body was spotted by an enemy soldier or a surveillance camera.
When an alarm sounds, all the guards in the area are alerted to your presence. Alarms may also call in enemy reinforcements.
You can prevent an alarm from sounding by either not being spotted or hacking an alarm panel before it is triggered. Once an alarm has been triggered, you can turn it off by hacking an alarm panel, but it may be too late to prevent discovery by then.