Practical Sentinel: Reviewing and Optimizing costs

Staying on top of Azure consumption

Microsoft Sentinel is based on the Azure stack and thus billed as part of your Azure consumption. It is important to be wary of the monthly cost as it can quickly expand if it is not being watched carefully.

Microsoft Sentinel is billed in two ways:

• Ingestion: This is the biggest chunk in the Sentinel budget. Data that is added into Sentinel is billed, with a few exceptions.
• Retention: By default, there is a 90-day free retention period for Microsoft Sentinel. If you want to retain data for longer than 90 days, you will be billed. The maximum retention is 12 years.

Before diving into the content at hand, there are a couple of important notes:

• If the tenant has Microsoft 365 E5 licensing, it will receive a daily ingestion discount for specific sources.
• When there are virtual machines that have a Defender for Servers plan 2, there is a 500 MB free ingestion per machine on specific data sources.

Gaining Insights

There are two main ways to gain insights into the current cost of Microsoft Sentinel: The Azure Cost Analysis and Sentinel workbooks.

The Azure Cost Analysis view is the only way to get correct details on your actual spend, as this is where the bill is based upon. Navigate to the resource group where Microsoft Sentinel is located and select cost analysis. Afterwards, select ‘Cost by resource’ at the top of the page. Within this list, search for the resource type ‘Log Analytics workspace’. If this resource is expanded, the Microsoft Sentinel cost will be visible as seen in Figure 1.

This view is useful as you can see the different categories the cost is built up from: What is data ingestion, data retention and potential discounts (benefits) because of licenses.

This method does not provide a means to identify which data connector is contributing the most to the overall cost. For this, we need to retrieve the information from Microsoft Sentinel itself. The best way to do this is using the cost workbook. To get the cost workbook, execute the following steps as seen in Figure 2:

1. Open the Content Hub.
2. Search for the Microsoft Sentinel Cost workbook in the desired valuta.
3. Click install.
4. After the installation, it is available in the workbooks blade, within the template tab.

Within this workbook, a graphic ‘Breakdown by log category’ is available. An example can be seen in Figure 3. This breakdown is very insightful as you can immediately see what the logs are costing you the most. Based on this data, we can start looking for optimizations.

How is the Data Being used

Each table requires specific questions to be addressed:

• Are Analytic Rules configured on the data source?
• Is it being queried?
• How ‘old’ are the logs that are being used.

To verify whether the data is utilized within an analytic rule, it is necessary to determine if the table is referenced in a KQL query. There is no native way to do this. My suggestion is to export all analytic rules to ARM templates and search for the table name using a tool such as Notepad++ or Visual Studio code. This way, you can easily search text across multiple files. If a table isn’t used in analytic rules, you should think about moving it to a lower tier storage, which is covered in the next section.

If the data isn’t used in analytic rules, the next step is to check if the data is being used. There are two options to verify this: Either you ask this in the team or use the Log Analytic Audit Query events. This contains all of the queries that are run through the portal or API, but it needs to be enabled manually. By using a simple KQL query, you can see how much a table has been queried in the past 30 days. If the logs aren’t used, you should check whether the logs need to be retained at all. A sample query can be found below.

LAQueryLogs
| where TimeGenerated > ago(30d)
| where QueryText contains "SigninLogs"
| count

The final step is to verify the age of the logs being used. Sentinel provides 90 days of data retention for free; every extra day is billed additionally.  If data is retained longer for that but it’s not being used, you can check whether the retention period can be updated for that table.

Lower Tier Storage

Microsoft Sentinel has four different storage types:

• Analytic logs
• Basic logs
• Auxiliary logs
• Archive logs

Analytic logs are the default log type that supports all Microsoft Sentinel use cases, but it’s also the most expensive one.

Basic logs has been available since 2022 but are being phased out in favor of auxiliary logs which is currently in preview. This log type is much cheaper and is meant for high-volume logs, examples are firewall and proxy logs. However, they cannot be used in analytic rules and querying them is not free.

If logs are needed for investigations but not used in analytic rules; you should look into converting these into auxiliary logs. This will greatly decrease the cost, while still allowing the data to be used during investigations.

Updating the Retention Period

Updating the retention is a tricky thing to do. Having a long log retention can be seen as an ‘insurance’ as we hope to never use it. But when you need it, you are happy that you have it.

Choosing the correct retention period is dictated by compliance requirements, laws, internal decisions and costs. For more details, check out my previous article ‘Choosing an Appropriate Retention Period for Microsoft Sentinel Workspaces’.

When working with retention periods, ensure to also incorporate the archive log tier. Archives are cheaper than analytic logs for retention, but requires you to restore it before it can be used.

Commitment Tiers

When working in large Sentinel deployments, Microsoft offers discounts for environments that ensure minimum ingestion. This is what is called a commitment tier. They start at 100GB per day and offer steep discounts. This discount starts at 32% and goes up to 53% when ingesting more than 50 000 GB’s per day.

To know if it is interesting to enable a commitment tier, it is advised to use the ‘Workspace Usage Report’.  To find the optimal commitment tier, go to Cost Analysis > Microsoft Sentinel. This can be seen in Figure 4.

Frequency is Required

Cost management is not a one-off action, but it requires continuous monitoring. It is an important process to implement within a SOC team and ensure it is continuously being run. There are multiple aspects within the platform and each aspect is important. By doing so, we can ensure the SIEM is healthy and isn’t bloated in terms of pricing.