設定

Alertmanager 是透過命令列旗標和設定檔進行設定的。雖然命令列旗標設定不可變的系統參數,但設定檔定義了抑制規則、通知路由和通知接收器。

視覺化編輯器 可以協助建構路由樹。

若要檢視所有可用的命令列旗標,請執行 alertmanager -h

Alertmanager 可以在執行階段重新載入其設定。如果新的設定格式不正確,則變更將不會被套用並記錄錯誤。傳送 SIGHUP 至該程序或傳送 HTTP POST 要求至 /-/reload 端點,即可觸發設定重新載入。

設定檔簡介

若要指定要載入的設定檔,請使用 --config.file 旗標。

./alertmanager --config.file=alertmanager.yml

該檔案以 YAML 格式撰寫,由以下描述的結構定義。括號表示參數為選用。對於非清單參數,該值會設定為指定的預設值。

一般預留位置定義如下

  • <duration>:符合正規表示式 ((([0-9]+)y)?(([0-9]+)w)?(([0-9]+)d)?(([0-9]+)h)?(([0-9]+)m)?(([0-9]+)s)?(([0-9]+)ms)?|0) 的持續時間,例如 1d1h30m5m10s
  • <labelname>:符合正規表示式 [a-zA-Z_][a-zA-Z0-9_]* 的字串
  • <labelvalue>:Unicode 字元字串
  • <filepath>:目前工作目錄中的有效路徑
  • <boolean>:可以採用值 truefalse 的布林值
  • <string>:一般字串
  • <secret>:屬於機密的正規字串,例如密碼
  • <tmpl_string>:使用前已進行範本展開的字串
  • <tmpl_secret>:使用前已進行範本展開且為機密的字串
  • <int>:整數值
  • <regex>:任何有效的 RE2 正規表示式(正規表示式會固定在兩端。若要取消固定正規表示式,請使用 .*<regex>.*。)

其他預留位置會單獨指定。

提供的有效範例檔案會顯示內容中的用法。

檔案配置和全域設定

全域設定指定在所有其他設定內容中有效的參數。它們也可以作為其他設定區段的預設值。其他頂層區段記錄在本頁下方。

global:
  # The default SMTP From header field.
  [ smtp_from: <tmpl_string> ]
  # The default SMTP smarthost used for sending emails, including port number.
  # Port number usually is 25, or 587 for SMTP over TLS (sometimes referred to as STARTTLS).
  # Example: smtp.example.org:587
  [ smtp_smarthost: <string> ]
  # The default hostname to identify to the SMTP server.
  [ smtp_hello: <string> | default = "localhost" ]
  # SMTP Auth using CRAM-MD5, LOGIN and PLAIN. If empty, Alertmanager doesn't authenticate to the SMTP server.
  [ smtp_auth_username: <string> ]
  # SMTP Auth using LOGIN and PLAIN.
  [ smtp_auth_password: <secret> ]
  # SMTP Auth using LOGIN and PLAIN.
  [ smtp_auth_password_file: <string> ]
  # SMTP Auth using PLAIN.
  [ smtp_auth_identity: <string> ]
  # SMTP Auth using CRAM-MD5.
  [ smtp_auth_secret: <secret> ]
  # The default SMTP TLS requirement.
  # Note that Go does not support unencrypted connections to remote SMTP endpoints.
  [ smtp_require_tls: <bool> | default = true ]

  # The API URL to use for Slack notifications.
  [ slack_api_url: <secret> ]
  [ slack_api_url_file: <filepath> ]
  [ victorops_api_key: <secret> ]
  [ victorops_api_key_file: <filepath> ]
  [ victorops_api_url: <string> | default = "https://alert.victorops.com/integrations/generic/20131114/alert/" ]
  [ pagerduty_url: <string> | default = "https://events.pagerduty.com/v2/enqueue" ]
  [ opsgenie_api_key: <secret> ]
  [ opsgenie_api_key_file: <filepath> ]
  [ opsgenie_api_url: <string> | default = "https://api.opsgenie.com/" ]
  [ wechat_api_url: <string> | default = "https://qyapi.weixin.qq.com/cgi-bin/" ]
  [ wechat_api_secret: <secret> ]
  [ wechat_api_corp_id: <string> ]
  [ telegram_api_url: <string> | default = "https://api.telegram.org" ]
  [ webex_api_url: <string> | default = "https://webexapis.com/v1/messages" ]
  # The default HTTP client configuration
  [ http_config: <http_config> ]

  # ResolveTimeout is the default value used by alertmanager if the alert does
  # not include EndsAt, after this time passes it can declare the alert as resolved if it has not been updated.
  # This has no impact on alerts from Prometheus, as they always include EndsAt.
  [ resolve_timeout: <duration> | default = 5m ]

# Files from which custom notification template definitions are read.
# The last component may use a wildcard matcher, e.g. 'templates/*.tmpl'.
templates:
  [ - <filepath> ... ]

# The root node of the routing tree.
route: <route>

# A list of notification receivers.
receivers:
  - <receiver> ...

# A list of inhibition rules.
inhibit_rules:
  [ - <inhibit_rule> ... ]

# DEPRECATED: use time_intervals below.
# A list of mute time intervals for muting routes.
mute_time_intervals:
  [ - <mute_time_interval> ... ]

# A list of time intervals for muting/activating routes.
time_intervals:
  [ - <time_interval> ... ]

與路由相關的設定可讓您設定如何根據時間路由、彙總、限制和靜音警示。

<route>

路由區塊定義路由樹中的節點及其子節點。其選用設定參數會從其父節點繼承 (如果未設定)。

每個警示都會在設定的頂層路由進入路由樹,此路由必須符合所有警示 (即未設定任何匹配器)。然後,它會遍歷子節點。如果 continue 設定為 false,則會在第一個符合的子節點之後停止。如果符合的節點上的 continue 為 true,則警示會繼續比對後續的同層節點。如果警示不符合節點的任何子節點 (沒有符合的子節點,或不存在子節點),則警示會根據目前節點的設定參數進行處理。

如需關於群組的詳細資訊,請參閱Alertmanager 概念

[ receiver: <string> ]
# The labels by which incoming alerts are grouped together. For example,
# multiple alerts coming in for cluster=A and alertname=LatencyHigh would
# be batched into a single group.
#
# To aggregate by all possible labels use the special value '...' as the sole label name, for example:
# group_by: ['...']
# This effectively disables aggregation entirely, passing through all
# alerts as-is. This is unlikely to be what you want, unless you have
# a very low alert volume or your upstream notification system performs
# its own grouping.
[ group_by: '[' <labelname>, ... ']' ]

# Whether an alert should continue matching subsequent sibling nodes.
[ continue: <boolean> | default = false ]

# DEPRECATED: Use matchers below.
# A set of equality matchers an alert has to fulfill to match the node.
match:
  [ <labelname>: <labelvalue>, ... ]

# DEPRECATED: Use matchers below.
# A set of regex-matchers an alert has to fulfill to match the node.
match_re:
  [ <labelname>: <regex>, ... ]

# A list of matchers that an alert has to fulfill to match the node.
matchers:
  [ - <matcher> ... ]

# How long to initially wait to send a notification for a group
# of alerts. Allows to wait for an inhibiting alert to arrive or collect
# more initial alerts for the same group. (Usually ~0s to few minutes.)
# If omitted, child routes inherit the group_wait of the parent route.
[ group_wait: <duration> | default = 30s ]

# How long to wait before sending a notification about new alerts that
# are added to a group of alerts for which an initial notification has
# already been sent. (Usually ~5m or more.) If omitted, child routes
# inherit the group_interval of the parent route.
[ group_interval: <duration> | default = 5m ]

# How long to wait before sending a notification again if it has already
# been sent successfully for an alert. (Usually ~3h or more). If omitted,
# child routes inherit the repeat_interval of the parent route.
# Note that this parameter is implicitly bound by Alertmanager's
# `--data.retention` configuration flag. Notifications will be resent after either
# repeat_interval or the data retention period have passed, whichever
# occurs first. `repeat_interval` should be a multiple of `group_interval`.
[ repeat_interval: <duration> | default = 4h ]

# Times when the route should be muted. These must match the name of a
# mute time interval defined in the mute_time_intervals section.
# Additionally, the root node cannot have any mute times.
# When a route is muted it will not send any notifications, but
# otherwise acts normally (including ending the route-matching process
# if the `continue` option is not set.)
mute_time_intervals:
  [ - <string> ...]

# Times when the route should be active. These must match the name of a
# time interval defined in the time_intervals section. An empty value
# means that the route is always active.
# Additionally, the root node cannot have any active times.
# The route will send notifications only when active, but otherwise
# acts normally (including ending the route-matching process
# if the `continue` option is not set).
active_time_intervals:
  [ - <string> ...]

# Zero or more child routes.
routes:
  [ - <route> ... ]

範例

# The root route with all parameters, which are inherited by the child
# routes if they are not overwritten.
route:
  receiver: 'default-receiver'
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 4h
  group_by: [cluster, alertname]
  # All alerts that do not match the following child routes
  # will remain at the root node and be dispatched to 'default-receiver'.
  routes:
  # All alerts with service=mysql or service=cassandra
  # are dispatched to the database pager.
  - receiver: 'database-pager'
    group_wait: 10s
    matchers:
    - service=~"mysql|cassandra"
  # All alerts with the team=frontend label match this sub-route.
  # They are grouped by product and environment rather than cluster
  # and alertname.
  - receiver: 'frontend-pager'
    group_by: [product, environment]
    matchers:
    - team="frontend"

  # All alerts with the service=inhouse-service label match this sub-route.
  # the route will be muted during offhours and holidays time intervals.
  # even if it matches, it will continue to the next sub-route
  - receiver: 'dev-pager'
    matchers:
      - service="inhouse-service"
    mute_time_intervals:
      - offhours
      - holidays
    continue: true

    # All alerts with the service=inhouse-service label match this sub-route
    # the route will be active only during offhours and holidays time intervals.
  - receiver: 'on-call-pager'
    matchers:
      - service="inhouse-service"
    active_time_intervals:
      - offhours
      - holidays

<time_interval>

time_interval 指定時間的具名間隔,可在路由樹中參照此間隔,以在一天中的特定時間靜音/啟用特定路由。

name: <string>
time_intervals:
  [ - <time_interval_spec> ... ]

<time_interval_spec>

time_interval_spec 包含時間間隔的實際定義。語法支援下列欄位

- times:
  [ - <time_range> ...]
  weekdays:
  [ - <weekday_range> ...]
  days_of_month:
  [ - <days_of_month_range> ...]
  months:
  [ - <month_range> ...]
  years:
  [ - <year_range> ...]
  location: <string>

所有欄位都是清單。在每個非空清單中,必須滿足至少一個元素才能符合該欄位。如果欄位未指定,則任何值都會符合該欄位。若要使時間的瞬時符合完整的時間間隔,則所有欄位都必須符合。某些欄位支援範圍和負索引,詳細說明如下。如果未指定時區,則時間將以 UTC 為準。

time_range:範圍包含開始時間,但不包含結束時間,以便輕鬆表示在小時邊界開始/結束的時間。例如,start_time: '17:00'end_time: '24:00' 將於 17:00 開始,並在 24:00 之前立即結束。它們指定如下

    times:
    - start_time: HH:MM
      end_time: HH:MM

weekday_range:一週中幾天的清單,其中一週從星期日開始,到星期六結束。應按名稱指定日期 (例如「星期日」)。為方便起見,也接受 <start_day>:<end_day> 形式的範圍,且兩端都包含在內。例如:['monday:wednesday','saturday', 'sunday']

days_of_month_range:月份中數值的日期清單。日期從 1 開始。也接受從月底開始的負值,例如 1 月的 -1 表示 1 月 31 日。例如:['1:5', '-3:-1']。若延伸超過月份的開始或結束,則會將其固定。例如,在 2 月指定 ['1:31'] 會將實際結束日期固定為 28 或 29,視閏年而定。兩端都包含在內。

month_range:以不區分大小寫的名稱 (例如「January」) 或數字識別的日曆月份清單,其中 1 月 = 1。也接受範圍。例如:['1:3', 'may:august', 'december']。兩端都包含在內。

year_range:年度的數值清單。接受範圍。例如:['2020:2022', '2030']。兩端都包含在內。

location:符合 IANA 時區資料庫中位置的字串。例如:'Australia/Sydney'。位置會提供時間間隔的時區。例如,位置為 'Australia/Sydney' 的時間間隔包含類似

    times:
    - start_time: 09:00
      end_time: 17:00
    weekdays: ['monday:friday']

的內容,則會包含星期一至星期五,澳洲雪梨當地時間上午 9:00 到下午 5:00 之間的任何時間。

您也可以使用 'Local' 作為位置,以使用執行 Alertmanager 的機器的當地時間,或使用 'UTC' 表示 UTC 時間。如果未提供時區,則時間間隔將以 UTC 時間為準。注意: 在 Windows 上,除非您使用 ZONEINFO 環境變數提供自訂時區資料庫,否則僅支援 LocalUTC

抑制可讓您根據另一組警示的存在,將一組警示靜音。這樣可以建立系統或服務之間的相依性,以便在停機期間僅傳送一組相互關聯的警示中最相關的警示。

如需關於抑制的詳細資訊,請參閱Alertmanager 概念

<inhibit_rule>

抑制規則會在符合一組匹配條件的警報(目標)存在時,使符合另一組匹配條件的警報(來源)靜音。目標和來源警報都必須在 equal 列表中,具有相同標籤名稱的標籤值。

從語義上來說,缺少標籤和具有空值的標籤是相同的。因此,如果 equal 中列出的所有標籤名稱都從來源和目標警報中遺失,則抑制規則將會生效。

為了防止警報抑制自身,符合規則目標和來源兩邊的警報,不能被同樣符合兩邊條件的警報抑制(包括自身)。然而,我們建議選擇目標和來源的匹配條件時,應避免警報同時符合兩邊的條件。這樣比較容易理解,且不會觸發此特殊情況。

# DEPRECATED: Use target_matchers below.
# Matchers that have to be fulfilled in the alerts to be muted.
target_match:
  [ <labelname>: <labelvalue>, ... ]
# DEPRECATED: Use target_matchers below.
target_match_re:
  [ <labelname>: <regex>, ... ]

# A list of matchers that have to be fulfilled by the target
# alerts to be muted.
target_matchers:
  [ - <matcher> ... ]

# DEPRECATED: Use source_matchers below.
# Matchers for which one or more alerts have to exist for the
# inhibition to take effect.
source_match:
  [ <labelname>: <labelvalue>, ... ]
# DEPRECATED: Use source_matchers below.
source_match_re:
  [ <labelname>: <regex>, ... ]

# A list of matchers for which one or more alerts have
# to exist for the inhibition to take effect.
source_matchers:
  [ - <matcher> ... ]

# Labels that must have an equal value in the source and target
# alert for the inhibition to take effect.
[ equal: '[' <labelname>, ... ']' ]

標籤匹配條件

標籤匹配條件將警報與路由、靜音和抑制規則進行匹配。

重要:Prometheus 正在新增對標籤和指標中 UTF-8 的支援。為了在 Alertmanager 中也支援 UTF-8,Alertmanager 0.27 及更高版本針對匹配條件採用了新的剖析器,其中包含一些向後不相容的變更。雖然大多數匹配條件將會向前相容,但有些則不會。Alertmanager 目前處於過渡期,同時支援 UTF-8 和傳統匹配條件,並提供了一些工具來協助您為過渡做好準備。

如果這是新的 Alertmanager 安裝,我們建議在建立 Alertmanager 組態檔案之前,啟用 UTF-8 嚴格模式。您可以在此處找到如何啟用 UTF-8 嚴格模式的說明。

如果這是現有的 Alertmanager 安裝,我們建議在啟用 UTF-8 嚴格模式之前,先在稱為回退模式的預設模式下執行 Alertmanager。在此模式下,如果需要對組態檔案進行任何變更才能啟用 UTF-8 嚴格模式,Alertmanager 將會記錄警告。Alertmanager 將在接下來的兩個版本中將 UTF-8 嚴格模式設為預設模式,因此請務必盡快轉換。

無論 Alertmanager 安裝是新的還是現有的,您也可以使用 amtool 來驗證 Alertmanager 組態檔案是否與 UTF-8 嚴格模式相容,然後再於 Alertmanager 伺服器中啟用。您不需要執行中的 Alertmanager 伺服器即可執行此操作。您可以在此處找到如何使用 amtool 驗證 Alertmanager 組態檔案的說明。

Alertmanager 伺服器運作模式

在過渡期間,Alertmanager 支援三種運作模式。這些模式稱為回退模式、UTF-8 嚴格模式和傳統模式。回退模式是預設模式。

Alertmanager 伺服器的操作員應在過渡期結束前轉換為 UTF-8 嚴格模式。Alertmanager 將在接下來的兩個版本中將 UTF-8 嚴格模式設為預設模式,因此請務必盡快轉換。

回退模式

Alertmanager 在預設模式下以稱為回退模式的特殊模式執行。作為操作員,您不應感受到路由、靜音或抑制規則的運作方式有任何差異。

在回退模式下,組態會先剖析為 UTF-8 匹配條件,如果與 UTF-8 剖析器不相容,則會剖析為傳統匹配條件。如果您的 Alertmanager 組態包含與 UTF-8 剖析器不相容的匹配條件,Alertmanager 將會將其剖析為傳統匹配條件並記錄警告。此警告也包含關於如何將匹配條件從傳統匹配條件變更為 UTF-8 匹配條件的建議。例如

ts=2024-02-11T10:00:00Z caller=parse.go:176 level=warn msg="Alertmanager 正在移至用於標籤和匹配條件的新剖析器,且此輸入不相容。Alertmanager 改為使用傳統匹配條件剖析器做為回退。若要使此輸入與 UTF-8 匹配條件剖析器相容,請確保所有規則運算式和值都使用雙引號括住。如果您仍然看到此訊息,請開啟問題。" input="foo=" origin=config err="輸入結束:預期的標籤值" suggestion="foo=\"\""

在此範例中,可以將匹配條件 foo= 設為有效的 UTF-8 匹配條件,方法是將運算式右側用雙引號括住,產生 foo=""。這兩個匹配條件是等效的,但是使用 UTF-8 匹配條件時,匹配條件的右側是必填欄位。

在極少數情況下,組態可能會導致 UTF-8 和傳統剖析器之間出現不一致。當匹配條件在兩個剖析器中都有效時,就會發生這種情況,但由於新增了 UTF-8 支援,根據所使用的剖析器而導致不同的剖析結果。如果您的 Alertmanager 組態發生不一致,Alertmanager 將會使用傳統剖析器並記錄警告。例如

ts=2024-02-11T10:00:00Z caller=parse.go:183 level=warn msg="匹配條件輸入不一致" input="qux=\"\xf0\x9f\x99\x82\"\n" origin=config

應根據具體情況逐一檢查任何不一致的情況,因為根據不一致的性質,組態可能不需要在啟用 UTF-8 嚴格模式之前進行更新。例如,\xf0\x9f\x99\x82 是 🙂 表情符號的位元組序列。如果目的是比對文字的 🙂 表情符號,則不需要變更。但是,如果目的是比對文字的 \xf0\x9f\x99\x82,則應將匹配條件變更為 qux="\\xf0\\x9f\\x99\\x82"

UTF-8 嚴格模式

在 UTF-8 嚴格模式下,Alertmanager 會停用對傳統匹配條件的支援

alertmanager --config.file=config.yml --enable-feature="utf8-strict-mode"

新的 Alertmanager 安裝應啟用此模式,且現有的 Alertmanager 安裝在解決所有不相容的匹配條件警告後,也應啟用此模式。在解決所有不相容的匹配條件警告之前,Alertmanager 不會以 UTF-8 嚴格模式啟動

ts=2024-02-11T10:00:00Z caller=coordinator.go:118 level=error component=configuration msg="載入組態檔案失敗" file=config.yml err="輸入結束:預期的標籤值"

UTF-8 嚴格模式將在過渡期結束時成為 Alertmanager 的預設模式。

傳統模式

傳統模式等同於 Alertmanager 0.26.0 版和更舊版本

alertmanager --config.file=config.yml --enable-feature="classic-mode"

如果您懷疑回退模式或 UTF-8 嚴格模式有問題,可以使用此模式。在這種情況下,請在 GitHub 上開啟一個包含盡可能多資訊的問題。

驗證

您可以使用 amtool 來驗證 Alertmanager 組態檔案是否與 UTF-8 嚴格模式相容,然後再於 Alertmanager 伺服器中啟用。您不需要執行中的 Alertmanager 伺服器即可執行此操作。

就像 Alertmanager 伺服器一樣,如果組態不相容或包含不一致,amtool 將會記錄警告

amtool check-config config.yml
Checking 'config.yml'
level=warn msg="Alertmanager is moving to a new parser for labels and matchers, and this input is incompatible. Alertmanager has instead parsed the input using the classic matchers parser as a fallback. To make this input compatible with the UTF-8 matchers parser please make sure all regular expressions and values are double-quoted. If you are still seeing this message please open an issue." input="foo=" origin=config err="end of input: expected label value" suggestion="foo=\"\""
level=warn msg="Matchers input has disagreement" input="qux=\"\\xf0\\x9f\\x99\\x82\"\n" origin=config
  SUCCESS
Found:
 - global config
 - route
 - 2 inhibit rules
 - 2 receivers
 - 0 templates

amtool 中未記錄任何警告時,您就知道組態與 UTF-8 嚴格模式相容

amtool check-config config.yml
Checking 'config.yml'  SUCCESS
Found:
 - global config
 - route
 - 2 inhibit rules
 - 2 receivers
 - 0 templates

您也可以在 UTF-8 嚴格模式下使用 amtool 作為額外的驗證層級。您會知道組態無效,因為命令將會失敗

amtool check-config config.yml --enable-feature="utf8-strict-mode"
level=warn msg="UTF-8 mode enabled"
Checking 'config.yml'  FAILED: end of input: expected label value

amtool: error: failed to validate 1 file(s)

您會知道組態有效,因為命令將會成功

amtool check-config config.yml --enable-feature="utf8-strict-mode"
level=warn msg="UTF-8 mode enabled"
Checking 'config.yml'  SUCCESS
Found:
 - global config
 - route
 - 2 inhibit rules
 - 2 receivers
 - 0 templates

<matcher>

UTF-8 匹配條件

UTF-8 匹配條件由三個符號組成

  • 標籤名稱的未加引號的文字或雙引號字串。
  • =!==~!~ 其中之一。= 表示等於,!= 表示不等於,=~ 表示符合規則運算式,而 !~ 表示不符合規則運算式。
  • 規則運算式或標籤值的未加引號的文字或雙引號字串。

未加引號的文字可以包含除保留字元以外的所有 UTF-8 字元。這些保留字元是空格,以及 { } ! = ~ , \ " ' ` 中的所有字元。例如,foo[a-zA-Z]+Προμηθεύς (希臘文的 Prometheus) 都是有效的未加引號的文字範例。但是,foo! 不是有效的文字,因為 ! 是保留字元。

雙引號字串可以包含所有 UTF-8 字元。與未加引號的文字不同,沒有保留字元。您甚至可以使用 UTF-8 碼位。例如,"foo!""bar,baz""\"baz qux\"""\xf0\x9f\x99\x82" 都是有效的雙引號字串。

傳統匹配條件

傳統匹配條件是具有受 PromQL 和 OpenMetrics 啟發語法的字串。傳統匹配條件的語法包含三個符號

  • 有效的 Prometheus 標籤名稱。
  • =!==~!~ 其中之一。= 表示等於,!= 表示字串不相等,=~ 用於規則運算式的相等性,而 !~ 用於規則運算式的不相等性。它們的意義與 PromQL 選取器中的意義相同。
  • UTF-8 字串,可以用雙引號括住。在每個符號之前或之後,可以有任意數量的空格。

第 3 個符號可以是空字串。在第 3 個符號內,會套用 OpenMetrics 跳脫規則:\" 表示雙引號、\n 表示換行、\\ 表示文字反斜線。未跳脫的 " 不得在第 3 個符號內出現 (僅限作為第 1 個或最後一個字元)。但是,容許使用文字換行字元,以及後面沒有 \n" 的單個 \ 字元。在這種情況下,它們會作為文字反斜線。

匹配條件的組合

您可以組合匹配條件以建立複雜的匹配運算式。組合時,所有匹配條件都必須符合,整個運算式才能符合。例如,運算式 {alertname="Watchdog", severity=~"warning|critical"} 會符合帶有標籤 alertname=Watchdog, severity=critical 的警報,但不會符合帶有標籤 alertname=Watchdog, severity=none 的警報,因為雖然 alertname 為 Watchdog,但 severity 既不是 warning 也不是 critical。

您可以使用 YAML 清單將匹配條件組合成運算式

matchers:
  - alertname = Watchdog
  - severity =~ "warning|critical"

或者作為 PromQL 啟發的運算式,其中每個匹配條件都以逗號分隔

{alertname="Watchdog", severity=~"warning|critical"}

允許使用單一的尾隨逗號

{alertname="Watchdog", severity=~"warning|critical",}

開頭的 { 和結尾的 } 大括號是選用的

alertname="Watchdog", severity=~"warning|critical"

但是,必須同時存在或同時省略。您不能有不完整的開頭或結尾大括號

{alertname="Watchdog", severity=~"warning|critical"
alertname="Watchdog", severity=~"warning|critical"}

您也不能有重複的開頭或結尾大括號

{{alertname="Watchdog", severity=~"warning|critical",}}

允許在雙引號外使用空格(空格、定位點和換行符號),且不會對匹配條件本身產生任何影響。例如

{
   alertname = "Watchdog",
   severity =~ "warning|critical",
}

與以下項目等效

{alertname="Watchdog",severity=~"warning|critical"}

更多範例

以下是一些範例

  1. 兩個以 YAML 清單組合的等於匹配條件

    matchers:
      - foo = bar
      - dings != bums
    
  2. 兩個匹配器組合構成簡短的 YAML 列表

    matchers: [ foo = bar, dings != bums ]
    

如下所示,在簡短形式中,最好使用雙引號,以避免逗號等特殊字元的問題

   matchers: [ "foo = \"bar,baz\"", "dings != bums" ]
  1. 您也可以將兩個匹配器放入一個類似 PromQL 的字串中。單引號在這裡效果最好

    matchers: [ '{foo="bar", dings!="bums"}' ]
    
  2. 為了避免 YAML 中跳脫和引號規則的問題,您也可以使用 YAML 區塊

    matchers:
      - |
          {quote=~"She said: \"Hi, all!( How're you…)?\""}
    

這些接收器設定允許設定通知目的地(接收器)以及基於 HTTP 的接收器的 HTTP 用戶端選項。

<receiver>

接收器是一個或多個通知整合的命名設定。

注意:作為取消先前對新接收器暫緩措施的一部分,除了現有的要求之外,新的通知整合將需要有具有推送權限的專責維護者。

# The unique name of the receiver.
name: <string>

# Configurations for several notification integrations.
discord_configs:
  [ - <discord_config>, ... ]
email_configs:
  [ - <email_config>, ... ]
msteams_configs:
  [ - <msteams_config>, ... ]
opsgenie_configs:
  [ - <opsgenie_config>, ... ]
pagerduty_configs:
  [ - <pagerduty_config>, ... ]
pushover_configs:
  [ - <pushover_config>, ... ]
slack_configs:
  [ - <slack_config>, ... ]
sns_configs:
  [ - <sns_config>, ... ]
telegram_configs:
  [ - <telegram_config>, ... ]
victorops_configs:
  [ - <victorops_config>, ... ]
webex_configs:
  [ - <webex_config>, ... ]
webhook_configs:
  [ - <webhook_config>, ... ]
wechat_configs:
  [ - <wechat_config>, ... ]

<http_config>

http_config 允許設定接收器用來與基於 HTTP 的 API 服務通訊的 HTTP 用戶端。

# Note that `basic_auth` and `authorization` options are mutually exclusive.

# Sets the `Authorization` header with the configured username and password.
# password and password_file are mutually exclusive.
basic_auth:
  [ username: <string> ]
  [ password: <secret> ]
  [ password_file: <string> ]

# Optional the `Authorization` header configuration.
authorization:
  # Sets the authentication type.
  [ type: <string> | default: Bearer ]
  # Sets the credentials. It is mutually exclusive with
  # `credentials_file`.
  [ credentials: <secret> ]
  # Sets the credentials with the credentials read from the configured file.
  # It is mutually exclusive with `credentials`.
  [ credentials_file: <filename> ]

# Optional OAuth 2.0 configuration.
# Cannot be used at the same time as basic_auth or authorization.
oauth2:
  [ <oauth2> ]

# Whether to enable HTTP2.
[ enable_http2: <bool> | default: true ]

# Optional proxy URL.
[ proxy_url: <string> ]
# Comma-separated string that can contain IPs, CIDR notation, domain names
# that should be excluded from proxying. IP and domain names can
# contain port numbers.
[ no_proxy: <string> ]
# Use proxy URL indicated by environment variables (HTTP_PROXY, http_proxy, HTTPS_PROXY, https_proxy, NO_PROXY, and no_proxy)
[ proxy_from_environment: <boolean> | default: false ]
# Specifies headers to send to proxies during CONNECT requests.
[ proxy_connect_header:
  [ <string>: [<secret>, ...] ] ]

# Configure whether HTTP requests follow HTTP 3xx redirects.
[ follow_redirects: <bool> | default = true ]

# Configures the TLS settings.
tls_config:
  [ <tls_config> ]

<oauth2>

使用用戶端憑證授權類型的 OAuth 2.0 驗證。Alertmanager 從指定的端點使用給定的用戶端存取金鑰和密鑰取得存取權杖。

client_id: <string>
[ client_secret: <secret> ]

# Read the client secret from a file.
# It is mutually exclusive with `client_secret`.
[ client_secret_file: <filename> ]

# Scopes for the token request.
scopes:
  [ - <string> ... ]

# The URL to fetch the token from.
token_url: <string>

# Optional parameters to append to the token URL.
endpoint_params:
  [ <string>: <string> ... ]

# Configures the token request's TLS settings.
tls_config:
  [ <tls_config> ]

# Optional proxy URL.
[ proxy_url: <string> ]
# Comma-separated string that can contain IPs, CIDR notation, domain names
# that should be excluded from proxying. IP and domain names can
# contain port numbers.
[ no_proxy: <string> ]
# Use proxy URL indicated by environment variables (HTTP_PROXY, https_proxy, HTTPs_PROXY, https_proxy, and no_proxy)
[ proxy_from_environment: <boolean> | default: false ]
# Specifies headers to send to proxies during CONNECT requests.
[ proxy_connect_header:
  [ <string>: [<secret>, ...] ] ]

<tls_config>

tls_config 允許設定 TLS 連線。

# CA certificate to validate the server certificate with.
[ ca_file: <filepath> ]

# Certificate and key files for client cert authentication to the server.
[ cert_file: <filepath> ]
[ key_file: <filepath> ]

# ServerName extension to indicate the name of the server.
# http://tools.ietf.org/html/rfc4366#section-3.1
[ server_name: <string> ]

# Disable validation of the server certificate.
[ insecure_skip_verify: <boolean> | default = false]

# Minimum acceptable TLS version. Accepted values: TLS10 (TLS 1.0), TLS11 (TLS
# 1.1), TLS12 (TLS 1.2), TLS13 (TLS 1.3).
# If unset, Prometheus will use Go default minimum version, which is TLS 1.2.
# See MinVersion in https://pkg.go.dev/crypto/tls#Config.
[ min_version: <string> ]
# Maximum acceptable TLS version. Accepted values: TLS10 (TLS 1.0), TLS11 (TLS
# 1.1), TLS12 (TLS 1.2), TLS13 (TLS 1.3).
# If unset, Prometheus will use Go default maximum version, which is TLS 1.3.
# See MaxVersion in https://pkg.go.dev/crypto/tls#Config.
[ max_version: <string> ]

接收器整合設定

這些設定允許設定特定的接收器整合。

<discord_config>

Discord 通知透過 Discord webhook API 發送。請參閱 Discord 的 「Webhooks 簡介」文章,以了解如何為頻道設定 webhook 整合。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The Discord webhook URL.
# webhook_url and webhook_url_file are mutually exclusive.
webhook_url: <secret>
webhook_url_file: <filepath>

# Message title template.
[ title: <tmpl_string> | default = '{{ template "discord.default.title" . }}' ]

# Message body template.
[ message: <tmpl_string> | default = '{{ template "discord.default.message" . }}' ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<email_config>

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = false ]

# The email address to send notifications to.
to: <tmpl_string>

# The sender's address.
[ from: <tmpl_string> | default = global.smtp_from ]

# The SMTP host through which emails are sent.
[ smarthost: <string> | default = global.smtp_smarthost ]

# The hostname to identify to the SMTP server.
[ hello: <string> | default = global.smtp_hello ]

# SMTP authentication information.
# auth_password and auth_password_file are mutually exclusive.
[ auth_username: <string> | default = global.smtp_auth_username ]
[ auth_password: <secret> | default = global.smtp_auth_password ]
[ auth_password_file: <string> | default = global.smtp_auth_password_file ]
[ auth_secret: <secret> | default = global.smtp_auth_secret ]
[ auth_identity: <string> | default = global.smtp_auth_identity ]

# The SMTP TLS requirement.
# Note that Go does not support unencrypted connections to remote SMTP endpoints.
[ require_tls: <bool> | default = global.smtp_require_tls ]

# TLS configuration.
tls_config:
  [ <tls_config> ]

# The HTML body of the email notification.
[ html: <tmpl_string> | default = '{{ template "email.default.html" . }}' ]
# The text body of the email notification.
[ text: <tmpl_string> ]

# Further headers email header key/value pairs. Overrides any headers
# previously set by the notification implementation.
[ headers: { <string>: <tmpl_string>, ... } ]

<msteams_config>

Microsoft Teams 通知透過 傳入 Webhooks API 端點發送。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The incoming webhook URL.
# webhook_url and webhook_url_file are mutually exclusive.
[ webhook_url: <secret> ]
[ webhook_url_file: <filepath> ]

# Message title template.
[ title: <tmpl_string> | default = '{{ template "msteams.default.title" . }}' ]

# Message summary template.
[ summary: <tmpl_string> | default = '{{ template "msteams.default.summary" . }}' ]

# Message body template.
[ text: <tmpl_string> | default = '{{ template "msteams.default.text" . }}' ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<opsgenie_config>

OpsGenie 通知透過 OpsGenie API 發送。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The API key to use when talking to the OpsGenie API.
[ api_key: <secret> | default = global.opsgenie_api_key ]

# The filepath to API key to use when talking to the OpsGenie API. Conflicts with api_key.
[ api_key_file: <filepath> | default = global.opsgenie_api_key_file ]

# The host to send OpsGenie API requests to.
[ api_url: <string> | default = global.opsgenie_api_url ]

# Alert text limited to 130 characters.
[ message: <tmpl_string> | default = '{{ template "opsgenie.default.message" . }}' ]

# A description of the alert.
[ description: <tmpl_string> | default = '{{ template "opsgenie.default.description" . }}' ]

# A backlink to the sender of the notification.
[ source: <tmpl_string> | default = '{{ template "opsgenie.default.source" . }}' ]

# A set of arbitrary key/value pairs that provide further detail
# about the alert.
# All common labels are included as details by default.
[ details: { <string>: <tmpl_string>, ... } ]

# List of responders responsible for notifications.
responders:
  [ - <responder> ... ]

# Comma separated list of tags attached to the notifications.
[ tags: <tmpl_string> ]

# Additional alert note.
[ note: <tmpl_string> ]

# Priority level of alert. Possible values are P1, P2, P3, P4, and P5.
[ priority: <tmpl_string> ]

# Whether to update message and description of the alert in OpsGenie if it already exists
# By default, the alert is never updated in OpsGenie, the new message only appears in activity log.
[ update_alerts: <boolean> | default = false ]

# Optional field that can be used to specify which domain alert is related to.
[ entity: <tmpl_string> ]

# Comma separated list of actions that will be available for the alert.
[ actions: <tmpl_string> ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<responder>

# Exactly one of these fields should be defined.
[ id: <tmpl_string> ]
[ name: <tmpl_string> ]
[ username: <tmpl_string> ]

# "team", "teams", "user", "escalation" or "schedule".
type: <tmpl_string>

<pagerduty_config>

PagerDuty 通知透過 PagerDuty API 發送。PagerDuty 提供有關如何整合的文件。Alertmanager 的 v0.11 及更高版本對 PagerDuty Events API v2 的支援存在重要差異。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The routing and service keys are mutually exclusive.
# The PagerDuty integration key (when using PagerDuty integration type `Events API v2`).
# It is mutually exclusive with `routing_key_file`.
routing_key: <tmpl_secret>
# Read the Pager Duty routing key from a file.
# It is mutually exclusive with `routing_key`.
routing_key_file: <filepath>
# The PagerDuty integration key (when using PagerDuty integration type `Prometheus`).
# It is mutually exclusive with `service_key_file`.
service_key: <tmpl_secret>
# Read the Pager Duty service key from a file.
# It is mutually exclusive with `service_key`.
service_key_file: <filepath>

# The URL to send API requests to
[ url: <string> | default = global.pagerduty_url ]

# The client identification of the Alertmanager.
[ client:  <tmpl_string> | default = '{{ template "pagerduty.default.client" . }}' ]
# A backlink to the sender of the notification.
[ client_url:  <tmpl_string> | default = '{{ template "pagerduty.default.clientURL" . }}' ]

# A description of the incident.
[ description: <tmpl_string> | default = '{{ template "pagerduty.default.description" .}}' ]

# Severity of the incident.
[ severity: <tmpl_string> | default = 'error' ]

# Unique location of the affected system.
[ source: <tmpl_string> | default = client ]

# A set of arbitrary key/value pairs that provide further detail
# about the incident.
[ details: { <string>: <tmpl_string>, ... } | default = {
  firing:       '{{ template "pagerduty.default.instances" .Alerts.Firing }}'
  resolved:     '{{ template "pagerduty.default.instances" .Alerts.Resolved }}'
  num_firing:   '{{ .Alerts.Firing | len }}'
  num_resolved: '{{ .Alerts.Resolved | len }}'
} ]

# Images to attach to the incident.
images:
  [ <image_config> ... ]

# Links to attach to the incident.
links:
  [ <link_config> ... ]

# The part or component of the affected system that is broken.
[ component: <tmpl_string> ]

# A cluster or grouping of sources.
[ group: <tmpl_string> ]

# The class/type of the event.
[ class: <tmpl_string> ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<image_config>

這些欄位記錄在 PagerDuty API 文件中。

href: <tmpl_string>
src: <tmpl_string>
alt: <tmpl_string>

這些欄位記錄在 PagerDuty API 文件中。

href: <tmpl_string>
text: <tmpl_string>

<pushover_config>

Pushover 通知透過 Pushover API 發送。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The recipient user's key.
# user_key and user_key_file are mutually exclusive.
user_key: <secret>
user_key_file: <filepath>

# Your registered application's API token, see https://pushover.net/apps
# You can also register a token by cloning this Prometheus app:
# https://pushover.net/apps/clone/prometheus
# token and token_file are mutually exclusive.
token: <secret>
token_file: <filepath>

# Notification title.
[ title: <tmpl_string> | default = '{{ template "pushover.default.title" . }}' ]

# Notification message.
[ message: <tmpl_string> | default = '{{ template "pushover.default.message" . }}' ]

# A supplementary URL shown alongside the message.
[ url: <tmpl_string> | default = '{{ template "pushover.default.url" . }}' ]

# Optional device to send notification to, see https://pushover.net/api#device
[ device: <string> ]

# Optional sound to use for notification, see https://pushover.net/api#sound
[ sound: <string> ]

# Priority, see https://pushover.net/api#priority
[ priority: <tmpl_string> | default = '{{ if eq .Status "firing" }}2{{ else }}0{{ end }}' ]

# How often the Pushover servers will send the same notification to the user.
# Must be at least 30 seconds.
[ retry: <duration> | default = 1m ]

# How long your notification will continue to be retried for, unless the user
# acknowledges the notification.
[ expire: <duration> | default = 1h ]

# Optional time to live (TTL) to use for notification, see https://pushover.net/api#ttl
[ ttl: <duration> ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<slack_config>

Slack 通知可以透過傳入 WebhooksBot 權杖發送。

如果使用傳入 webhook,則 api_url 必須設定為傳入 webhook 的 URL,或寫入 api_url_file 中引用的檔案。

如果使用 Bot 權杖,則 api_url 必須設定為 https://slack.com/api/chat.postMessage,Bot 權杖必須在 http_config 中設定為授權憑證,並且 channel 必須包含要將通知傳送到的頻道名稱或頻道 ID。如果使用頻道名稱,則 # 是可選的。

通知包含一個附件

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = false ]

# The Slack webhook URL. Either api_url or api_url_file should be set.
# Defaults to global settings if none are set here.
[ api_url: <secret> | default = global.slack_api_url ]
[ api_url_file: <filepath> | default = global.slack_api_url_file ]

# The channel or user to send notifications to.
channel: <tmpl_string>

# API request data as defined by the Slack webhook API.
[ icon_emoji: <tmpl_string> ]
[ icon_url: <tmpl_string> ]
[ link_names: <boolean> | default = false ]
[ username: <tmpl_string> | default = '{{ template "slack.default.username" . }}' ]
# The following parameters define the attachment.
actions:
  [ <action_config> ... ]
[ callback_id: <tmpl_string> | default = '{{ template "slack.default.callbackid" . }}' ]
[ color: <tmpl_string> | default = '{{ if eq .Status "firing" }}danger{{ else }}good{{ end }}' ]
[ fallback: <tmpl_string> | default = '{{ template "slack.default.fallback" . }}' ]
fields:
  [ <field_config> ... ]
[ footer: <tmpl_string> | default = '{{ template "slack.default.footer" . }}' ]
[ mrkdwn_in: '[' <string>, ... ']' | default = ["fallback", "pretext", "text"] ]
[ pretext: <tmpl_string> | default = '{{ template "slack.default.pretext" . }}' ]
[ short_fields: <boolean> | default = false ]
[ text: <tmpl_string> | default = '{{ template "slack.default.text" . }}' ]
[ title: <tmpl_string> | default = '{{ template "slack.default.title" . }}' ]
[ title_link: <tmpl_string> | default = '{{ template "slack.default.titlelink" . }}' ]
[ image_url: <tmpl_string> ]
[ thumb_url: <tmpl_string> ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<action_config>

這些欄位記錄在 Slack API 文件中,關於訊息附件互動訊息

text: <tmpl_string>
type: <tmpl_string>
# Either url or name and value are mandatory.
[ url: <tmpl_string> ]
[ name: <tmpl_string> ]
[ value: <tmpl_string> ]

[ confirm: <action_confirm_field_config> ]
[ style: <tmpl_string> | default = '' ]
<action_confirm_field_config>

這些欄位記錄在Slack API 文件中。

text: <tmpl_string>
[ dismiss_text: <tmpl_string> | default '' ]
[ ok_text: <tmpl_string> | default '' ]
[ title: <tmpl_string> | default '' ]

<field_config>

這些欄位記錄在Slack API 文件中。

title: <tmpl_string>
value: <tmpl_string>
[ short: <boolean> | default = slack_config.short_fields ]

<sns_config>

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The SNS API URL i.e. https://sns.us-east-2.amazonaws.com.
#  If not specified, the SNS API URL from the SNS SDK will be used.
[ api_url: <tmpl_string> ]

# Configures AWS's Signature Verification 4 signing process to sign requests.
sigv4:
  [ <sigv4_config> ]

# SNS topic ARN, i.e. arn:aws:sns:us-east-2:698519295917:My-Topic
# If you don't specify this value, you must specify a value for the phone_number or target_arn.
# If you are using a FIFO SNS topic you should set a message group interval longer than 5 minutes
# to prevent messages with the same group key being deduplicated by the SNS default deduplication window
[ topic_arn: <tmpl_string> ]

# Subject line when the message is delivered to email endpoints.
[ subject: <tmpl_string> | default = '{{ template "sns.default.subject" .}}' ]

# Phone number if message is delivered via SMS in E.164 format.
# If you don't specify this value, you must specify a value for the topic_arn or target_arn.
[ phone_number: <tmpl_string> ]

# The  mobile platform endpoint ARN if message is delivered via mobile notifications.
# If you don't specify this value, you must specify a value for the topic_arn or phone_number.
[ target_arn: <tmpl_string> ]

# The message content of the SNS notification.
[ message: <tmpl_string> | default = '{{ template "sns.default.message" .}}' ]

# SNS message attributes.
attributes:
  [ <string>: <string> ... ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<sigv4_config>

# The AWS region. If blank, the region from the default credentials chain is used.
[ region: <string> ]

# The AWS API keys. Both access_key and secret_key must be supplied or both must be blank.
# If blank the environment variables `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are used.
[ access_key: <string> ]
[ secret_key: <secret> ]

# Named AWS profile used to authenticate.
[ profile: <string> ]

# AWS Role ARN, an alternative to using AWS API keys.
[ role_arn: <string> ]

<telegram_config>

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The Telegram API URL i.e. https://api.telegram.org.
# If not specified, default API URL will be used.
[ api_url: <string> | default = global.telegram_api_url ]

# Telegram bot token. It is mutually exclusive with `bot_token_file`.
[ bot_token: <secret> ]

# Read the Telegram bot token from a file. It is mutually exclusive with `bot_token`.
[ bot_token_file: <filepath> ]

# ID of the chat where to send the messages.
[ chat_id: <int> ]

# Message template.
[ message: <tmpl_string> default = '{{ template "telegram.default.message" .}}' ]

# Disable telegram notifications
[ disable_notifications: <boolean> | default = false ]

# Parse mode for telegram message, supported values are MarkdownV2, Markdown, HTML and empty string for plain text.
[ parse_mode: <string> | default = "HTML" ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<victorops_config>

VictorOps 通知透過 VictorOps API 發送。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The API key to use when talking to the VictorOps API.
# It is mutually exclusive with `api_key_file`.
[ api_key: <secret> | default = global.victorops_api_key ]

# Reads the API key to use when talking to the VictorOps API from a file.
# It is mutually exclusive with `api_key`.
[ api_key_file: <filepath> | default = global.victorops_api_key_file ]

# The VictorOps API URL.
[ api_url: <string> | default = global.victorops_api_url ]

# A key used to map the alert to a team.
routing_key: <tmpl_string>

# Describes the behavior of the alert (CRITICAL, WARNING, INFO).
[ message_type: <tmpl_string> | default = 'CRITICAL' ]

# Contains summary of the alerted problem.
[ entity_display_name: <tmpl_string> | default = '{{ template "victorops.default.entity_display_name" . }}' ]

# Contains long explanation of the alerted problem.
[ state_message: <tmpl_string> | default = '{{ template "victorops.default.state_message" . }}' ]

# The monitoring tool the state message is from.
[ monitoring_tool: <tmpl_string> | default = '{{ template "victorops.default.monitoring_tool" . }}' ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<webhook_config>

webhook 接收器允許設定通用接收器。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The endpoint to send HTTP POST requests to.
# url and url_file are mutually exclusive.
url: <secret>
url_file: <filepath>

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

# The maximum number of alerts to include in a single webhook message. Alerts
# above this threshold are truncated. When leaving this at its default value of
# 0, all alerts are included.
[ max_alerts: <int> | default = 0 ]

Alertmanager 將以下 JSON 格式的 HTTP POST 請求傳送到已設定的端點

{
  "version": "4",
  "groupKey": <string>,              // key identifying the group of alerts (e.g. to deduplicate)
  "truncatedAlerts": <int>,          // how many alerts have been truncated due to "max_alerts"
  "status": "<resolved|firing>",
  "receiver": <string>,
  "groupLabels": <object>,
  "commonLabels": <object>,
  "commonAnnotations": <object>,
  "externalURL": <string>,           // backlink to the Alertmanager.
  "alerts": [
    {
      "status": "<resolved|firing>",
      "labels": <object>,
      "annotations": <object>,
      "startsAt": "<rfc3339>",
      "endsAt": "<rfc3339>",
      "generatorURL": <string>,      // identifies the entity that caused the alert
      "fingerprint": <string>        // fingerprint to identify the alert
    },
    ...
  ]
}

有一個整合列表具有此功能。

<wechat_config>

WeChat 通知透過WeChat API 發送。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = false ]

# The API key to use when talking to the WeChat API.
[ api_secret: <secret> | default = global.wechat_api_secret ]

# The WeChat API URL.
[ api_url: <string> | default = global.wechat_api_url ]

# The corp id for authentication.
[ corp_id: <string> | default = global.wechat_api_corp_id ]

# API request data as defined by the WeChat API.
[ message: <tmpl_string> | default = '{{ template "wechat.default.message" . }}' ]
# Type of the message type, supported values are `text` and `markdown`.
[ message_type: <string> | default = 'text' ]
[ agent_id: <string> | default = '{{ template "wechat.default.agent_id" . }}' ]
[ to_user: <string> | default = '{{ template "wechat.default.to_user" . }}' ]
[ to_party: <string> | default = '{{ template "wechat.default.to_party" . }}' ]
[ to_tag: <string> | default = '{{ template "wechat.default.to_tag" . }}' ]

<webex_config>

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The Webex Teams API URL i.e. https://webexapis.com/v1/messages
# If not specified, default API URL will be used.
[ api_url: <string> | default = global.webex_api_url ]

# ID of the Webex Teams room where to send the messages.
room_id: <string>

# Message template.
[ message: <tmpl_string> default = '{{ template "webex.default.message" .}}' ]

# The HTTP client's configuration. You must use this configuration to supply the bot token as part of the HTTP `Authorization` header. 
[ http_config: <http_config> | default = global.http_config ]

這份文件是開源的。請透過提交問題或提取請求來協助改進它。