Agents Should Use All Available Data
A keynote by Alex Simons, Microsoft VP for Entra, at the recent TEC 2025 event caused me to reflect on the plans Microsoft has to help Entra administrators work smarter in an agentic future.
During his talk, Alex reported that the Microsoft tenant now supports nearly 125,000 agents, but didn’t say what kind of agents these are and how they’re created. The number is high enough for tenant administrators to wonder if agents really will run amok in the future. Microsoft’s response is that they’re working hard to make sure that agents can be managed with the same kind of attention to authentication, authorization, and auditing as apply to other Entra ID objects.
Alex offered examples of how Microsoft is using agents to help administrators. The initial set of agents created by Microsoft for Entra seem to attack the right problems and they are in preview rather than the finished version, but I think the agents can do more to exploit information that’s readily available.
Figuring Out Conditional Access
The Conditional Access optimization agent is a good example of how AI works well when provided with a solid knowledge base and limited target data set. In this case, the knowledge comes from the Entra developers, and the target data set is the conditional access policies in a tenant. Conditional access can be a real nightmare, and it’s all too easy for inexperienced administrators to mess things up, including to a point where every user is locked out of their account.
The agent reviews the set of conditional access policies to find security gaps and improve policy settings. It can also propose the consolidation of existing policies to simplify policy management. None of this is rocket science and an experienced Entra administrator who knows their way around conditional access could do the same work, but not quite at the speed that the agent can.
The administrators who interact with the agent must have Entra P1 licenses and the agent consumes Security Compute Units (SCUs), which is the basis for billing for Security Copilot. I think this agent will be popular with tenants feeling their way with conditional access or even in tenants with strong administrative talent who want a second opinion about the health of their conditional access policy inventory. In some cases, agent recommendations might be cross-checked against the output from Maester, the community-developed tool that does an excellent (and free) job of validating security configurations against best practice benchmarks.
While I liked the conditional access agent, I thought that that it could do more. When a free tool like Maester can do such a good job of highlighting issues with conditional access policies and much more in a tenant’s security configuration, you’d expect that AI-enhanced tools from Microsoft would be in a different class. Today, that’s not true, and I recommend that tenants invest time to master and use Maester before plunging into agentic checks. On the upside, I expect that the Entra agents will improve over time to justify the cost of both the additional licenses required for agents and the resources consumed by agents.
Making Access Reviews Smarter
An even more pronounced lack of ambition is present in the access review agent, which requires Entra ID governance or Entra Suite licenses (the maddening inconsistency in agent licensing is caused by following the requirements for human administrators). Once again, you’ll need to pay for SCUs to accommodate agent processing, with an access review of 20 decisions requiring 4.5 SCUs. Given the size of some access reviews, it’s easy to see how a lot of SCUs might be consumed (at $4 per “provisioned” SCU). That cost needs to come down to a more acceptable level before tenants will be interested in using agents to run access reviews.
Alex made the point that access reviews can become an exercise in clicking by administrators or group owners “just to get the review done.” That’s right, but I’m not sure that driving access reviews through Teams chat will make the reviews any more compelling.
My main issue is that the agent doesn’t go past the information used for manual access reviews when it makes its recommendations. This is a pity because there’s lots of information that experienced administrators know about that the agent could consult.
Take the example of an access review for membership of a Microsoft 365 group that contains sensitive information. The level of sensitivity can be conveyed by a container management sensitivity label. The higher the priority of the sensitivity label, the more sensitive the information managed by the group is likely to be. In addition, highly sensitive groups often inherit settings from container management labels that control how the group and linked applications behave. External sharing is an example.
Controlling the ability of group owners to add guests is an important group setting, and a sensitivity label can update the privacy settings for a group to block or allow guest members. Application of such a label only blocks or allows the addition of new guest members. Existing guests are left untouched. The agent doesn’t appear to take the privacy setting into account at present, but it seems to me that an access review should highlight guest members for removal from group membership if guests are present when blocked by group settings.
I also feel that the access review agent misses the opportunity to take a much more developed and holistic approach to reviewing group member activity. Assessing members using properties like their last sign in or whether the account is enabled is all very well, but some very rich information is available within Microsoft 365 that can help to assess member activity. For instance, the agent could check the unified audit log to discover what members generate content in a group by uploading files to or modifying files in the group’s SharePoint site. It could check on how many messages members post to the group mailbox or to channels in the team associated with the group. In other words, I expect the agent to use all available evidence to assess the suitability of someone to remain as a group member.
The Unified Audit Log Issue
Much of the information that an agent might consume is in the unified audit log. The problem is that the unified audit log is intended to be a tool for administrators to use when checking what happens inside a tenant. The log is certainly not designed to be a source of knowledge for agents that might need to check activity for hundreds of users during an access review. Perhaps I am wrong, and agents can interact with the unified audit log with aplomb. However, given recent efforts to move administrators from synchronous to asynchronous audit log searches, I can’t see how the audit log developers will be happy at the prospect of hyperactive agents coming along to run queries to discover evidence of user activity.
Perhaps the answer is for Microsoft to build a stripped-down, high-performance version of the unified audit log that only holds information relating to user activity, like SharePoint file operations. The agent-accessible log would have to be populated on a regular basis, just like Microsoft creates usage data for different workloads (always about 2 days behind real time).
Ambitious Agents Needed
The point is that agents need to do more than simply automate what happens today. If agents can only do what an experienced administrator can do, albeit faster, there’s little reason for customers to invest in the extra licenses and SCUs needed to run the agents.
Agents need to be more ambitious and exploit every scrap of information that might influence an outcome. If the data exists in a tenant, it should be used. If not, then the Entra agents will disappoint, just like the recently introduced Copilot administrative skills for SharePoint Online.
