Advanced Detectors

Historical Anomaly Detector

Objective #

Create a historical baseline anomaly detector using the detector wizard and examine the generated alert message.

Step 1 – Create the Detector #

Navigate to:

Alerts & Detectors → Create Detector → Custom Detector

ADD YOUR INITIALS before the proposed detector name.

Create Alert Rule

Configure the following in the alert signal:

• Signal (A): system.cpu.utilization

Add Filter

• Filter: deployment.environment : astronomy-shop

Proceed to Alert Condition, choose Historical Anomaly and then

Proceed to Alert Settings

• Cycle length: 1d
• Alert when: Too high
• Trigger Sensitivity: High

Show advanced settings and review

Proceed to Alert Message.

Step 2 – Examine the Default Alert Message #

Under Message Preview, click Customize and review the generated message:

{{#if anomalous}}
	Rule "{{{ruleName}}}" in detector "{{{detectorName}}}" triggered at {{timestamp}}.
{{else}}
	Rule "{{{ruleName}}}" in detector "{{{detectorName}}}" cleared at {{timestamp}}.
{{/if}}

{{#if anomalous}}
Triggering condition: {{{readableRule}}}
{{/if}}

Mean value of signal in the last {{event_annotations.current_window}}: {{inputs.summary.value}}
{{#if anomalous}}Trigger threshold: {{inputs.fire_top.value}}
{{else}}Clear threshold: {{inputs.clear_top.value}}.
{{/if}}

{{#notEmpty dimensions}}
Signal details:
{{{dimensions}}}
{{/notEmpty}}

{{#if anomalous}}
{{#if runbookUrl}}Runbook: {{{runbookUrl}}}{{/if}}
{{#if tip}}Tip: {{{tip}}}{{/if}}
{{/if}}

{{#if detectorTags}}Tags: {{detectorTags}}{{/if}}

{{#if detectorTeams}}
Teams:{{#each detectorTeams}} {{name}}{{#unless @last}},{{/unless}}{{/each}}.
{{/if}}

What This Message Is Doing #

This message uses conditional blocks to render different content depending on whether the detector is triggering or clearing.

• {{#if anomalous}} renders content only when the detector is firing.
• The {{else}} branch renders when the detector clears.

This allows one template to handle both trigger and clear notifications.

Important Variables Available in Alert Messages #

The following variables are automatically available:

• {{ruleName}} – Name of the alert rule
• {{detectorName}} – Name of the detector
• {{timestamp}} – Time of the event
• {{readableRule}} – Human-readable firing condition
• {{event_annotations.current_window}} – Evaluation window duration
• {{inputs.summary.value}} – Aggregated metric value for the evaluation window
• {{inputs.fire_top.value}} – Historical anomaly trigger threshold
• {{inputs.clear_top.value}} – Historical anomaly clear threshold
• {{dimensions}} – Dimension key/value pairs (host, environment, etc.)
• {{runbookUrl}} – Configured runbook link (if set)
• {{tip}} – Configured tip (if set)
• {{detectorTags}} – Tags assigned to the detector
• {{detectorTeams}} – Assigned teams

Any stream that is published in SignalFlow becomes available as: {{inputs.<stream_name>.value}}

Click Done Editing to close the custom message.

Proceed to Alert Recipients and do not select anything, we don’t actually want to send notifications for this scenario

Proceed to Alert Activation

Activate Alert Rule

When prompted about Missing Alert Notification Policy, choose Save