Why You Should Conduct Regular Entra ID Assessments

Benefits of Validating Your Entra ID Configuration

Entra ID is a fundamental part of Microsoft 365 and one of the most important areas to get right in tenant management. It’s important to validate the Entra ID configuration periodically to ensure that changes that happen over time don’t cause issues. One of the main challenges organizations face to ensure that their directory operates according to best practices is to find the time to understand what a “good” configuration is and compare those settings with what they have in place.

As a Microsoft partner, I often work with customers to help them understand and assess gaps within Microsoft 365. Several tools are available to help assess a Microsoft 365 environment such as the Conditional Access assessment report and the Office 365 ATP Recommended Configuration Analyzer (ORCA). When it comes to assessing Entra ID, one of the best tools to use in my opinion is the Azure AD Assessment created by Merill Fernando, Principal Product Manager for Microsoft Entra.

In this article, I explain what the Azure AD Assessment is and how it can assess and report on your Entra ID configuration, as well as plan the remediation of any discovered issues. As with any templated recommendations, some of the recommendations may incorporate features that require enhanced licensing, so it is important to take all recommendations in the context of your environment.

What is the Azure AD Assessment?

The Azure AD Assessment (yes, it has not been renamed to Entra yet) is a tool containing interview-based assessments, scripts to gather data, Power BI reports, and PowerPoint slide decks to communicate the outputs. The assessment comes with a detailed guide to help you run the assessment in any Entra environment.

While the assessment Wiki outlines that it is designed for Microsoft employees or partners to run, nothing in the process prevents an administrator from running the tool for their tenant. The assessment aims to provide actionable outputs to guide administrators to improve their configuration and close any identifiable gaps.

Running the Assessment

The assessment wiki page contains a detailed explanation of the technical steps and any prerequisites or permissions required for data gathering, so I won’t review those details here. At a high level, the assessment is broken into three phases:

• Kick-off meeting and pre-interview data gathering
• Interview
• Closure Report, recommendations, and next steps

Kick-off Meeting and Pre-interview Data Gathering

The initial phase of the assessment sets the scene for the stakeholders involved in the assessment and gathers the raw data needed for the next steps. Using the prepared PowerPoint template, assemble those involved and run through the goals and expectations for the assessment. This meeting should set expectations of what to expect from the assessment and what is required from them.

After the kick-off, the tenant administrator should know how to run the data-gathering script. The details are available from the same GitHub repository. The assessment includes steps to gather data from Entra ID and, if required, Microsoft Entra Connect and ADFS. For the Entra assessment, Graph APIs gather data, and both delegated and application authentication are supported. This is important as having both delegated and application authentication available allows the flexibility to run the assessment while adhering to an organization’s security policy. For example, many organizations will not grant the permissions required to run the assessment to a user account, but prefer to manage the permission assignments through an app registration which can be restricted and removed after the assessment.

After gathering the data, you can complete this part of the assessment to generate the following outputs:

• AzureADAssessment.pbit – This Power BI report contains details of key components of the Entra configuration including details of:
  • Assigned admin roles
  • Tenant notification settings
  • Permission assignments to Entra registered apps
  • Expiration details for registered app secrets and certificates
  • Consent grants to registered apps

This detail can be invaluable when assessing an Entra tenant and can highlight some fundamental issues such as permanently assigned roles or over-permissioned apps. Figure 1 shows an example of the quality of the output within the assessment report.

• AzureADAssessment-ConditionalAccess.pbit – This Power BI report contains details specifically related to Conditional Access configuration. The Conditional Access configuration is validated against common Microsoft best practices. While Microsoft’s best practices may not be relevant to every organization, understanding the differences in your configuration to what Microsoft recommends is still a positive step. The report contains a tab for each of the following:
  • Proper Exclusion – Validates that exclusions from policies in your environment are minimized. Key components of this check are validating that break glass accounts are excluded from all conditional access policies as outlined in the break glass hygiene check and exclusions are managed by group membership rather than individual user accounts.
  • Proper Inclusion – Validates that inclusions in policies are based on either roles or groups rather than individual user accounts.
  • Break Glass Hygiene – Validates that there are two common Global Administrator accounts excluded from all policies. These accounts should be Break Glass accounts and configured with the appropriate protections such as an extremely complex password and alerting for sign-ins.
  • Block Legacy Authentication – Validates a policy exists that prevents legacy authentication across the tenant. There can be legitimate reasons to have legacy authentication allowed for particular accounts but as a rule, this should be minimized.
  • Bad Practice Avoidance – Validates that there are well-known settings that are likely to cause problems in the tenant. For example, a policy that may lock out all users from accessing the tenant.
  • App Lockout Risk – Validates if there are specific policies in the tenant which block access to all apps.
  • Guest Coverage – Validates that Guest users are appropriately targeted by Conditional Access policies.
  • Office 365 Coverage – Validates that the “Office 365” app is used in policies in place of the individual component apps within Office 365. The Office 365 app is available as a target app in conditional access and is used to target all Office 365 services with a single assignment. The full list of services included in the Office 365 app is listed here.
  • Azure Management Coverage – Validates that the “Microsoft Azure Management” app is protected by Conditional Access policies. This is commonly missed when configuring Conditional Access for Microsoft 365 workloads. The Microsoft Azure Management app is important to consider in conditional access as the Azure admin center is not included in the Office 365 app mentioned above.
  • Stale Inclusion/Exclusion – Validates if Conditional Access Policies contain deleted users, groups or apps. These should be removed to ensure the policies only contain the appropriate identities.
  • Network Locations – Validates the use of named and trusted locations in the tenant to either block access from specific locations or mark locations as trusted.

Figure 2 shows a sample output from the home page of the Conditional Access Power BI report. This page gives an overview of where issues may have been found and marks them for review. Depending on your configuration and requirements, some of the recommendations in the report may not be valid for you, but it’s important to understand the recommendation and document why it doesn’t apply in your scenario.

• OrganizationName_AADConnectSync_report.html contains full export of the Entra Connect configuration if Entra Connect is in place. This report can be filtered to identify any changes from the default configuration and is used when generating recommendations around any identity sync issues.
• ADFS to AAD App Migration Report Template.xlsm contains a detailed breakdown of applications that use ADFS if it is in place. This report contains recommendations for any issues with existing configurations and an assessment of which apps can be migrated to Microsoft Entra ID from ADFS. This report uses the ADFS to Azure AD App Migration Tool which is available separately.

Interview-based Assessment

With the technical discovery complete, the next step is to carry out the interview with the appropriate stakeholders. The assessment guide identifies these stakeholders as:

• Identity and Access Management (IAM) Architect
• Identity and Access Management (IAM) Operations
• InfoSec Architect
• InfoSec Operations

Every organization is different though so you may find that this list doesn’t match your organization’s structure. The key point here is you need to engage with personnel who possess both an architecture and operational perspective for identity management and security to arrive at an effective view of the environment and procedures.

Using the Azure AD Assessment Interview Worksheet, carry out the interview with the group over a recorded session so you can revisit the recording when compiling the final report. In the meeting, you review the Guided Walkthrough tab and the Interview Questions tab. The Guided Walkthrough takes you through the Microsoft Entra ID admin center and poses questions around the configuration while the Interview Questions look deeper into processes and policy.

Once the interview is complete, complete the Post-Interview assessment section using the data gathered in the previous step. This finishes the data-gathering phase of the assessment.

Closure Report, Recommendations, and Next Steps

To close out the assessment, take all of the gathered information and populate the output report PowerPoint template with the configuration and recommendations identified. The Azure AD Assessment Reference page can be used to add context to recommendations made. It’s critical to include the organization context in the outputs of the assessment, following Microsoft recommendations is nice but for many organizations, this won’t address the nuances of their requirements.

The output report should be used to identify and prioritize the recommendations being made and define clear goals, actions, and owners for the next steps. When presenting the report, it’s important to plan the next steps based on the assessment. There’s no point running the assessment if you’re not going to action it going forward!

A Worthwhile Endeavour

I mentioned that the assessment is meant to help Microsoft Partners and employees assess customer environments. However, many tenant administrators are more than capable of running the assessment for their own environment.

Whether you run the assessment or have a trusted partner do it, it’s an extremely useful process to validate your existing configuration and plan for future improvements. As Microsoft 365 is a fundamental part of the IT infrastructure for many organizations, having a solid foundation in Entra ID is one of the most important contributions to success.