What is automated alert triage?
Automated alert triage is software doing the first pass on every alert: how noisy it is, how it relates to the others firing, and what to look at first. It happens as the alert arrives, so the people who are paged start with a reason, and noise can be kept off the page.
What is automated alert triage?
Triage is the first look at an alert: is it real, how urgent is it, and what is it connected to? Often a tired person does it at 3 a.m. Automated triage does the same first look in software, for every alert, as it arrives. The result is a score, the reasons behind it, and a decision about what should reach a person.
Why automate it?
Alert noise costs attention. Each page that turns out to be nothing teaches people to trust the pager a little less, and the real one arrives into that doubt. Triage by software means the noisy alerts are read and sorted by something that does not get tired, and the pages that remain are the ones that matter. The 30-day plan for alert fatigue shows how a team works through the same problem by hand.
How does Noci AI triage alerts?
| Step | What happens |
|---|---|
| AI triage alongside paging | From the first alert, AI triage runs alongside paging, so the page and the work start together |
| Triage before anyone is paged | You can set up triage and rules to run before escalation |
| A noise score, with the reasons | Each alert carries a noise score, with the reasons shown beside it |
| Noisy alerts kept off a policy | You can keep alerts above a flakiness score off an escalation policy |
In Noci AI, AI triage runs alongside paging from the first alert, so the page does not wait for it. You can also set up triage and rules to run before escalation, which is triage before anyone is paged. Paging itself runs through your escalation policies and the channels they use: phone, SMS, Slack channel and Microsoft Teams.
What should you ask of any triage tool?
- Does it show its reason? A score with no reasons cannot be checked or argued with.
- Where do suppressed alerts go? They should stay visible, with the reason, so a rule that is wrong can be found.
- Who sets the limit? Look for a threshold you control rather than a fixed one.
- What does it do with the alerts that remain? The best result is a page that arrives with context. See suppression by policy for rules a human can defend, and deduplication, correlation and suppression for the differences.
How do you tell it is working?
Judge triage by what your team feels and by what you can count in your own history, not by a vendor’s claim. Look at:
- Pages that needed no action. Fewer is the goal, and the remainder should have a reason you can read.
- Alerts kept off a page that later mattered. Each one is a rule or a score to adjust.
- How often people override it. Frequent overrides mean the reasons are not persuasive.
- How long it takes to reach the first useful fact. Triage should shorten the gap between the alert and a person who knows why it fired.
What does triage not do?
It does not decide the cause of an incident, and it does not replace the people who are on call. It decides what is worth a person’s attention and shows why. What happens next is set by your escalation policy and your team.
Questions
Is automated triage the same as muting alerts?
No. Muting hides an alert by a rule you wrote. Triage reads each alert, scores it and shows the reasons, and can then keep a noisy alert off an escalation policy. The reason stays visible, so a person can check it.
Does triage delay paging?
No. In Noci AI, AI triage runs alongside paging from the first alert, so the page does not wait for it. You can also set triage and rules to run before anyone is paged. Paging goes out by phone, SMS, Slack channel and Microsoft Teams, as your escalation policy sets.
What is a noise score?
A score on each alert for how noisy it is, with the reasons shown beside it. In Noci AI, a noisy alert can be kept off an escalation policy by its flakiness score.
Can I see why an alert was kept off a page?
Yes. Each alert carries its noise score and the reasons, so the decision can be read and challenged.
Does triage replace on-call?
No. It decides what reaches the person on call and with what context. On-call schedules and escalation policies still decide who is paged.