Is Artificial Intelligence for Microsoft 365 Usable in Your Tenant?
Based on customer reaction, the reveal of a $30/user monthly subscription cost for Microsoft 365 Copilot came as a surprise. Most felt that the price was too high for their organization to consider deploying Copilot, even if Microsoft discounts the cost in some enterprise agreements. Deploying Copilot at $360 per user annually or $1,080 over a three-year agreement is a lot to heap on top of existing license costs, even if restricted to selected users.
A feeling emerged that the line of sight to results justifying the cost is not yet evident enough to commit to deployment. Others felt that it was a pity that Microsoft had not opted for a lower price to encourage widespread adoption. The price is the price, and now Microsoft 365 tenants have to figure out if using Copilot is a realistic prospect for them. With this goal in mind, the immediate questions to answer are:
- Who in our organization could use Copilot to increase their productivity in a measurable sense?
- How can productivity be measured? Do we know what it looks like, and is it possible to report increased productivity across the organization to understand the return on investment (ROI)?
Microsoft 365 Business Standard and Business Premium are the entry-level licenses for Microsoft 365 Copilot. These licenses are available to small to medium organizations of up to 300 users. In this discussion, I focus on enterprise tenants, meaning that the eligible licenses are Microsoft 365 E3 and E5.
Selecting Microsoft 365 Copilot Users
Every organization is different. The work is different. Company goals are different. People are different. But one thing’s for sure, deciding to assign a Copilot license to every account is a fool’s errand that will only lead to overspending and underuse. $360 a year is a lot to pay for a license assigned to someone who doesn’t use the technology, or uses Copilot to generate slightly better emails or create a list of tasks from a couple of meetings. The question here is how much the organization is willing to pay for these kinds of outputs. My bet is “not much.” Achieving real results with measurable benefits should be the aim of any license assignment exercise.
The granular license management available in Microsoft 365 allows the assignment of tailored licenses to users. Not everyone needs a Microsoft 365 E5 license, nor do they need a range of add-on licenses for solutions like Viva Topics. It’s important to match the assigned licenses with someone’s work responsibilities so that people have the right tools to do their job.
Understanding the characteristics of the kind of jobs that might use Copilot effectively is a starting point for deciding who should get these licenses. According to Microsoft, Copilot is best used by someone who works with “an abundance of data” stored in SharePoint Online, OneDrive for Business, Exchange Online, and Teams. In other words, knowledge workers that depend on Microsoft 365.
People store lots of information in Microsoft 365 in emails, meetings, Office documents (Word, Excel, PowerPoint), and PDFs. These are the basic objects that Copilot can query using the Graph APIs as it responds to user prompts (queries). For instance, Figure 1 shows Copilot being used in Word to draft a proposal based on some meeting notes. Copilot has found some documents that might help, all of which are stored in a SharePoint Online site or OneDrive for Business account accessible to the user.
If a role depends on information stored in other repositories and doesn’t make extensive use of Microsoft 365, they’re unlikely to get a lot of benefit from Copilot.
Because Microsoft 365 Copilot depends on access to information stored in Microsoft 365 repositories, it makes sense to draw up a set of jobs as a guide for license assignment. Accept that the list will be imperfect and that some people with jobs outside the set will prove their ability to generate beneficial results from Copilot. Statements like those listed below are the kind of guidelines that could be used to focus on jobs where Copilot might help:
- Writes reports based on analysis of large amounts of information stored.
- Produces customer-facing material like proposals or product information.
- Creates comparisons of different options available for projects or other decisions.
- Generates project updates on a regular basis.
The sine qua non factor is that the information people depend on to get their job done must be stored in Microsoft 365.
Not Everyone Qualifies for Microsoft 365
Not everyone with the same job title or function might be able to use Copilot. Not everyone in the same department might be able to, either. It’s also clear that Copilot will need special handling in some countries where unions or worker’s councils must be consulted before the introduction of new technology, especially when some believe that artificial intelligence will lead to making workers redundant in some areas.
From a management perspective, difficulty can arise from people who aren’t assigned a Copilot license when they want to have one (or feel that they should have one). Assigning licenses on a “want” basis rather than a proven need leads to an avoidable licensing overspend. It’s important to communicate that Copilot is a tool (albeit one embedded in all the Office apps). If you don’t need a tool to do your job, why would you have it?
Being able to show how people use Copilot effectively in the context of your organization is a good way to prove to someone that they don’t need the tool. All sorts of stories will circulate about the triumphs of artificial intelligence over mundane day-to-day work and how Copilot saves hours of effort for the fortunate few who can use it. You don’t have a problem if the stories are true and the results they report can be proven because Copilot is obviously very successful within the organization. It’s time to order more licenses and bring freedom to the downtrodden who don’t have Copilot licenses today. But before doing so, check and verify that the reported results are true and can be replicated elsewhere.
Measuring the Impact of Microsoft 365 Copilot
Measuring the success of any software requires effort. In the case of Microsoft 365 Copilot, consider these questions:
- How to measure the quality of Copilot output versus what a user is capable of on their own.
- How to measure time savings. Are the savings real, and will they persist on an ongoing basis? In other words, Copilot might be enormously helpful in refining a couple of paragraphs in an important report to make the text more concise and convincing. What value do you put on this output? And will every report be enhanced similarly with the same economic value?
- What does success look like?
The value of time saved depends on the cost of the user’s work. For instance, if a lawyer finishes a case submission a half-hour faster because Copilot helped them to craft the text and is able to bill another customer $500 for that half-hour, the annual $360 cost of Copilot becomes a non-event, even if the saving happens only once or twice a year. Remember that savings need to be achieved for every user (or at least a large majority of users assigned Copilot licenses) to justify Copilot deployment within the organization. One great example of achieving extra productivity and savings does not create a successful project!
Given that Copilot is not yet in public preview, determining the success of a deployment is guesswork. I suggest that three areas could be measured:
- What Users Do: The day-to-day experience of users assigned Copilot licenses. Are they using Copilot across all of Office or just for certain tasks? What are those tasks, and why is Copilot successful there and not elsewhere?
- What Users Achieve: Are users happy with the results gained with Copilot? Do they have to spend a lot of time adjusting the results to remove mistakes (always possible with artificial intelligence) and clarify matters? Measuring better outcomes in reports, documents, presentations, and the like is harder than assessing the same when programmers use tools like Copilot for GitHub.
- The Impact on User Output: Can people point to time savings generated by Copilot that result in obvious economic payback for the organization? Do users think they are more productive, and why do they think they are?
The questions asked could differ across business units. For instance, you could ask salespeople about the quality of customer documentation that they generate and if it helps them to close sales faster with less effort. Positive feedback from salespeople should be backed up by a matching increase in sales performance measured against the baseline before the introduction of Copilot. If sales don’t increase, then the application of AI to customer documentation has had no impact.
Run a Microsoft 365 Copilot Pilot
A pilot project is a classic step in IT deployments. The pilot validates that the software under test does what it’s supposed to and delivers the anticipated results. Given the newness of Microsoft 365 Copilot and the truth that few outside Microsoft understand its effectiveness in production, it makes sense to run a Copilot pilot.
One way to approach a pilot is to assign a small number of Copilot licenses to people in several departments who are acknowledged “knowledge generators.” By this, I mean people responsible for creating artifacts like company reports. See how they use Copilot and the results they achieve after the newness of the digital assistant wears off. Understand if they discover new ways of working enabled by Copilot and what tips and techniques they’ve found that can be shared with coworkers. A solid pilot helps an organization discover how and why the technology works for its people. That information should be captured and communicated to users after the pilot finishes.
Start of a Journey to a World of AI Tools
Microsoft 365 Copilot marks the start of a journey to a world where tools built using artificial intelligence are commonplace. In five years’ time, we will probably look back at text like this and wonder why we were so bothered.
It would be wonderful if Microsoft expanded its horizon by making Copilot available to Office 365 E3 and E5 tenants, even at the current elevated price. It would be even better if they dropped the cost of a Copilot license to a more reasonable level. After all, consider the value delivered by an Office 365 E3 license at $23/month. Now ask if $30/month is reasonable for Copilot. I still have difficulty with that price level, but I don’t get to decide. I look forward to Microsoft proving that the price is right when they finally allow me to use Copilot.
TEC Talk: Making Generative AI Work for Microsoft 365 Active Directory
AI-based Microsoft 365 Copilot is coming. But before you commit, join this TEC Talk to understand the technology behind Copilot and how it generates information from M365 applications.