ElastALert

We will be learning about the elastalert elastalert framework. Elastalert is used for alerting any unusual spikes or any pattern that needs to be alerted.

We need to make sure that API response time doesn't exceed a certain value. So to set a limit for the proper analysis the API response time must be calculated for a period of time say for a month and it is recommended to calculate its value at 95 percentile. Once this is done now the APIs response time must be calculated at every 5-10 minutes of interval.

MODULE CREATION

Create a module with the directory __init__.py there the packages are written. The function add_data lets you download all the documents that are matching in your query 10000 at a time.

If the data size is large use use_count_query it will override the global set limit.

Example of how it can be done

Go to the elasticsearch home directory present in /opt/elasticsearch

Create a new directory for the modules.Now we just need to write the config rules in the file.

We just need to learn abouth the run_every, buffer_time, and time_frame.run_every and buffer_time are fixed,the are defined already.we just need to understand their working. run_every lets the user to configure at what intervals the user rule will run. buffer_time can be understood in docs, “ElastAlert will continuously query against a window from the present to buffer_time ago.” It meens  buffer_time greater must be than the run_every as ElastAlert doesn’t repeat queries on the same time duration.We just need to store older required information from the queries if we want to use it in the alert logic repeatedly. time_frame is a variable it can be manupulated in the subclasses. You can use this variable to perform any action and obtain results on the documents lying within a certain time period.

realert:  This prevents you from getting repeated alerts if the same alert occurs multiple times in the same query run, it will give just one alert. We must set it to 0.

Also Read: Understanding Elk Stack

below is the alert config file

name: "POD ALERT"
index: "your inder"
type: "you type"
percentile_value: "we defined it 95"
run_every:
  hours : "your haour"
buffer_time:
  hours : "your hour"
realert:
  minutes: 0
max_query_size: 10000
max_scrolling_count: 0
  
filter:
- query:
    query_string: "pattern you want to filter"

alert:
- slack
slack:
slack_webhook_url: "slac url"
slack_username_override: "username"
slack_channel_override: "#percentile-alert"
slack_emoji_override: "custom emoji:"

We are seasoned DevOps solutions and service providers with vast experience in providing full-scale DevOps solutions for varied business requirements. Our team of DevOps professionals formulates effective strategies to strengthen your enterprise IT infrastructure and enhance operational efficiency. Our 360-degree DevOps solutions and services accelerate the software development lifecycle and ensure faster delivery with continuous deployment. For project related queries, reach us out at [email protected]