Microsoft Launches Copilot Agents Without Lifecycle Support
In September 2024, Satya Nadella ushered in Copilot Wave 2, including the introduction of Copilot agents to “automate and execute business processes.” Users could create agents in Copilot Studio and SharePoint Online. Since then, agents have proliferated, driven by an incessant drumbeat from Microsoft marketing to convince customers about the benefits of agents. The latest comment that might chill the hearts of tenant administrators is Nadella’s remark to industry analysts after Microsoft’s FY25 Q2 results that “With Copilot Studio, we are making it as simple to build an agent as it is to create an Excel spreadsheet.”
Certainly, Microsoft has put considerable effort into developing tools to build and distribute agents. Low-code development environments like the Teams Toolkit make it easy to create declarative agents, while Copilot Studio is available to work on more complex creations. All seems well and agents appear to be on the launchpad for success.
Administration is the Missing Link
But then we discover that tools to manage the lifecycle of agents are largely missing. Although it’s true that the deployment of apps to users is managed through the Installed Apps section of the Microsoft 365 admin center, and a report detailing agent usage is available in the reporting section, that’s about the limit for management functionality.
Some will respond that the lack of management is natural at this stage in the evolution of agent technology. After all, we’re at the height of the hype cycle and should be exploring the possibilities of the benefits that AI agents can generate for an organization instead of worrying about how these components can be managed.
Such an attitude is understandable for those who work in marketing. Their task is to stoke the fires of hype and drive customer interest with tales about the wonders of artificial intelligence. The text they write is uncritical and focuses on benefits. The stories they weave are full of positivity and joy. It’s not in their nature to worry about system administration.
Let’s consider the management needs to control agents in an organization of 50,000 users. How many agents might be created over a three-year period? How many of these agents remain in active use six months after their creation? What happens when those who create agents leave the business? How many agents are duplicates or near duplicates?
Given that we’re in the early days of agents, we don’t have good answers to these questions. Better knowledge will come in time as the technical community gains experience and develops answers.
The Same Story Rolls Out Again
We’ve heard this story before. Microsoft has an unfortunate habit of launching technologies without adequate administrative tools. Customers are still struggling with the legacy of Exchange public folders (1996). A more recent example is the lack of attention given to Teams administration in the early days after its 2017 launch. The legacy of the Teams fiasco is felt by organizations attempting to control the documents Copilot for Microsoft 365 can access. Microsoft development has always focused on delivering new user-facing features than the more mundane and prosaic capabilities that make tenant administration more bearable.
The Current State of Agent Administration
The Microsoft 365 admin center has a Copilot section where Microsoft presents all the settings and other information relevant to Copilot (Figure 1), like the pay-as-you-go settings to allow people without Microsoft 365 Copilot licenses to use agents on a metered (consumption) basis.
In this case, the setting we’re interested in controls the users allowed to use Copilot extensions (agents). According to Microsoft documentation, the setting to control agents will be available until February 12, 2025, after which I assume Microsoft will deliver another method to control who can access and run Copilot agents.
Disabling access or restricting access to specific users means that blocked accounts cannot access agents, and that third-party agents are disabled. If completely disabled, administrators cannot manage Copilot agents in the Integrated Apps section. Users won’t be able to load new agents, and published agents are no longer be visible in apps like BizChat. It can take up to an hour for an updated setting to be respected in clients.
Some organizations have elected to disable agents entirely until they are confident that adequate tools are available to manage agents, especially at scale. For instance, making an agent available to users through Installed Apps depends on agent developers submitting agents (apps) for approval by tenant administrators. Once approved, users can load and interact with agents. The process works, but I have doubts as to how well administrators could cope with the workload generated by having to review the details of agents submitted for publication (Figure 2).
Reviewing and approving a couple of apps isn’t hard. Doing the same for several hundred (or perhaps thousands) of agents annually is a different matter, especially if administrators need to check details like agent capabilities (like the external and internal sites used by the agent). Just tracking who’s creating what agents and who can use those agents for what purpose becomes challenging. Today’s agents are relatively simple, but as agents gain new capabilities, the time required to review and approve agents will increase. No one wants to approve an agent that isn’t 100%.
Some tender loving care also needs to be devoted to the way that custom agents are presented to users. For instance, the Get Agents option for BizChat in my tenant displays a bunch of agents “built for your organization” that are obsolete vestiages of deleted agents that I created when experimenting with declarative agents development (Figure 3).
One (O365 IT Pros V2) is valid. Another (O365BlogsDev) shows up in the Integrated Apps section of the Microsoft 365 admin center as “available” but not deployed. I was able to block that app, but there’s no trace of the remaining two agents and I have no idea how to clean up the phantoms. The problem could be linked to sideloading development copies of agents into BizChat, but it would be nice if the system could clean itself out periodically.
Making Copilot Agents Manageable
Complaining is easy. Coming up with solutions to solve a problem is a lot harder. In terms of making Copilot agents more manageable so that we don’t end up in the same kind of mess that organizations have experienced with public folders and teams, here are some ideas for functionality that Microsoft could implement to ameliorate potential future problems:
- Allow organizations to implement policy-driven management for agents to restrict the ability to create and use agents in communities defined by groups (including dynamic groups).
- Implement an agent expiration system that removes agents that aren’t run by a user for a specified period (for example, 90 days). The removed agents might enter an archived state for a period before permanent removal.
- Amend the Microsoft 365 delete user account process so that any agents owned by a user can be transferred to someone else, like what happens with OneDrive accounts. Although it does a good job covering the basics, the account deletion process doesn’t deal with “no code” artifacts associated with user accounts like Power Automate flows and agents. Some upgrades are needed in this area.
- Implement AI-powered checks when creating an agent to detect if the new agent will duplicate the functionality
- Deliver better reporting about the creation of agents, their usage, and agents that have not been used recently. The current Copilot usage API isn’t granular enough and doesn’t make details of agent usage available (a report about agent usage is available in the Microsoft 365 admin center).
- Make APIs available to enable organizations to manage agents proactively, including removing agents deemed unnecessary. If the APIs are comprehensive enough, tenants can develop solutions to meet their needs.
I’m sure that anyone who has spent any time dealing with agent creation, publication, and management will have their own ideas about agent management. If you have an idea, please note it as a comment.
Avoiding Disaster
It’s natural that people will be enthusiastic about new functionality. This applies both to creators and consumers. However, experience of previous new technologies and how they developed informs us that good management capabilities are an essential pillar of success. I fear that the proliferation of agents in Microsoft 365 will lay the foundation for future woe if Microsoft doesn’t address the current administration deficit. And that would be a real pity.
