Is Artificial Intelligence for Microsoft 365 Usable in Your Tenant?
Based on customer reaction, the reveal of a $30/user monthly subscription cost for Microsoft 365 Copilot came as a surprise. Most felt that the price was too high for their organization to consider deploying Copilot, even if Microsoft discounts the cost in some enterprise agreements. Deploying Copilot at $360 per user annually or $1,080 over a three-year agreement is a lot to heap on top of existing license costs, even if restricted to selected users.
A feeling emerged that the line of sight to results justifying the cost is not yet evident enough to commit to deployment. Others felt that it was a pity that Microsoft had not opted for a lower price to encourage widespread adoption. The price is the price, and now Microsoft 365 tenants have to figure out if using Copilot is a realistic prospect for them. With this goal in mind, the immediate questions to answer are:
- Who in our organization could use Copilot to increase their productivity in a measurable sense?
- How can productivity be measured? Do we know what it looks like, and is it possible to report increased productivity across the organization to understand the return on investment (ROI)?
Microsoft 365 Business Standard and Business Premium are the entry-level licenses for Microsoft 365 Copilot. These licenses are available to small to medium organizations of up to 300 users. In this discussion, I focus on enterprise tenants, meaning that the eligible licenses are Office 365 E3 and E5 and Microsoft 365 E3 and E5.
Update: See this article about using Microsoft 365 usage data to gain an insight into account activity across the Microsoft 365 workloads.
Selecting Microsoft 365 Copilot Users
Every organization is different. The work is different. Company goals are different. People are different. But one thing’s for sure, deciding to assign a Copilot license to every account is a fool’s errand that will only lead to overspending and underuse. $360 a year is a lot to pay for a license assigned to someone who doesn’t use the technology, or uses Copilot to generate slightly better emails or create a list of tasks from a couple of meetings. The question here is how much the organization is willing to pay for these kinds of outputs. My bet is “not much.” Achieving real results with measurable benefits should be the aim of any license assignment exercise.
The granular license management available in Microsoft 365 allows the assignment of tailored licenses to users. Not everyone needs a Microsoft 365 E5 license, nor do they need a range of add-on licenses for solutions like Viva Topics. It’s important to match the assigned licenses with someone’s work responsibilities so that people have the right tools to do their job.
Understanding the characteristics of the kind of jobs that might use Copilot effectively is a starting point for deciding who should get these licenses. According to Microsoft, Copilot is best used by someone who works with “an abundance of data” stored in SharePoint Online, OneDrive for Business, Exchange Online, and Teams. In other words, knowledge workers that depend on Microsoft 365.
People store lots of information in Microsoft 365 in emails, meetings, Office documents (Word, Excel, PowerPoint), and PDFs. These are the basic objects that Copilot can query using the Graph APIs as it responds to user prompts (queries). For instance, Figure 1 shows Copilot being used in Word to draft a proposal based on some meeting notes. Copilot has found some documents that might help, all of which are stored in a SharePoint Online site or OneDrive for Business account accessible to the user.
If a role depends on information stored in other repositories and doesn’t make extensive use of Microsoft 365, they’re unlikely to get a lot of benefit from Copilot.
Because Microsoft 365 Copilot depends on access to information stored in Microsoft 365 repositories, it makes sense to draw up a set of jobs as a guide for license assignment. Accept that the list will be imperfect and that some people with jobs outside the set will prove their ability to generate beneficial results from Copilot. Statements like those listed below are the kind of guidelines that could be used to focus on jobs where Copilot might help:
- Writes reports based on analysis of large amounts of information stored.
- Produces customer-facing material like proposals or product information.
- Creates comparisons of different options available for projects or other decisions.
- Generates project updates on a regular basis.
The sine qua non factor is that the information people depend on to get their job done must be stored in Microsoft 365.
Not Everyone Qualifies for Microsoft 365
Not everyone with the same job title or function might be able to use Copilot. Not everyone in the same department might be able to, either. It’s also clear that Copilot will need special handling in some countries where unions or worker’s councils must be consulted before the introduction of new technology, especially when some believe that artificial intelligence will lead to making workers redundant in some areas.
From a management perspective, difficulty can arise from people who aren’t assigned a Copilot license when they want to have one (or feel that they should have one). Assigning licenses on a “want” basis rather than a proven need leads to an avoidable licensing overspend. It’s important to communicate that Copilot is a tool (albeit one embedded in all the Office apps). If you don’t need a tool to do your job, why would you have it?
Being able to show how people use Copilot effectively in the context of your organization is a good way to prove to someone that they don’t need the tool. All sorts of stories will circulate about the triumphs of artificial intelligence over mundane day-to-day work and how Copilot saves hours of effort for the fortunate few who can use it. You don’t have a problem if the stories are true and the results they report can be proven because Copilot is obviously very successful within the organization. It’s time to order more licenses and bring freedom to the downtrodden who don’t have Copilot licenses today. But before doing so, check and verify that the reported results are true and can be replicated elsewhere.
Measuring the Impact of Microsoft 365 Copilot
Measuring the success of any software requires effort. In the case of Microsoft 365 Copilot, consider these questions:
- How to measure the quality of Copilot output versus what a user is capable of on their own.
- How to measure time savings. Are the savings real, and will they persist on an ongoing basis? In other words, Copilot might be enormously helpful in refining a couple of paragraphs in an important report to make the text more concise and convincing. What value do you put on this output? And will every report be enhanced similarly with the same economic value?
- What does success look like?
The value of time saved depends on the cost of the user’s work. For instance, if a lawyer finishes a case submission a half-hour faster because Copilot helped them to craft the text and is able to bill another customer $500 for that half-hour, the annual $360 cost of Copilot becomes a non-event, even if the saving happens only once or twice a year. Remember that savings need to be achieved for every user (or at least a large majority of users assigned Copilot licenses) to justify Copilot deployment within the organization. One great example of achieving extra productivity and savings does not create a successful project!
Given that Copilot is not yet in public preview, determining the success of a deployment is guesswork. I suggest that three areas could be measured:
- What Users Do: The day-to-day experience of users assigned Copilot licenses. Are they using Copilot across all of Office or just for certain tasks? What are those tasks, and why is Copilot successful there and not elsewhere?
- What Users Achieve: Are users happy with the results gained with Copilot? Do they have to spend a lot of time adjusting the results to remove mistakes (always possible with artificial intelligence) and clarify matters? Measuring better outcomes in reports, documents, presentations, and the like is harder than assessing the same when programmers use tools like Copilot for GitHub.
- The Impact on User Output: Can people point to time savings generated by Copilot that result in obvious economic payback for the organization? Do users think they are more productive, and why do they think they are?
The questions asked could differ across business units. For instance, you could ask salespeople about the quality of customer documentation that they generate and if it helps them to close sales faster with less effort. Positive feedback from salespeople should be backed up by a matching increase in sales performance measured against the baseline before the introduction of Copilot. If sales don’t increase, then the application of AI to customer documentation has had no impact.
Run a Microsoft 365 Copilot Pilot
A pilot project is a classic step in IT deployments. The pilot validates that the software under test does what it’s supposed to and delivers the anticipated results. Given the newness of Microsoft 365 Copilot and the truth that few outside Microsoft understand its effectiveness in production, it makes sense to run a Copilot pilot.
One way to approach a pilot is to assign a small number of Copilot licenses to people in several departments who are acknowledged “knowledge generators.” By this, I mean people responsible for creating artifacts like company reports. See how they use Copilot and the results they achieve after the newness of the digital assistant wears off. Understand if they discover new ways of working enabled by Copilot and what tips and techniques they’ve found that can be shared with coworkers. A solid pilot helps an organization discover how and why the technology works for its people. That information should be captured and communicated to users after the pilot finishes.
Start of a Journey to a World of AI Tools
Microsoft 365 Copilot marks the start of a journey to a world where tools built using artificial intelligence are commonplace. In five years’ time, we will probably look back at text like this and wonder why we were so bothered.
It would be wonderful if Microsoft expanded its horizon by making Copilot available to Office 365 E3 and E5 tenants, even at the current elevated price. It would be even better if they dropped the cost of a Copilot license to a more reasonable level. After all, consider the value delivered by an Office 365 E3 license at $23/month. Now ask if $30/month is reasonable for Copilot. I still have difficulty with that price level, but I don’t get to decide. I look forward to Microsoft proving that the price is right when they finally allow me to use Copilot.
TEC Talk: Making Generative AI Work for Microsoft 365 Active Directory
AI-based Microsoft 365 Copilot is coming. But before you commit, join this TEC Talk to understand the technology behind Copilot and how it generates information from M365 applications.
I would be interested how the game of licensing will change when the free windows copilot comes into scope ? I guess the hankering around getting a license for Copilot will reduce drastically in a company when they start benefitting already with Windows Copilot ( which is free). Thought?
Windows Copilot has nothing to do with Microsoft 365 Copilot. There’s nothing remotely similar as far as I can see. I doubt the presence of Windows Copilot will have any effect on Microsoft pricing strategy.
$100k to run the test for 300 users is really only a 8k savings…. not really something to jump at 🙂
Our organization was offered Copilot early access. $100K for 300 licenses and we accepted. Waiting to be onboarded. A lot of the points you mention in the article are things we are considering around quantifying the value of Copilot. I can almost certainly say that $30 pupm worth of value isn’t there unless you are a heavy note taker in meetings like a project manager or sales rep.
Hi Nick,
Thanks for confirming the rate Microsoft is looking to extract from customers to participate in the Copilot test. I guess we’ll all learn soon enough where the value can be extracted from.
Hi Nick,
Thanks for confirming that you are in the same situation as us. 2 follow up questions:
1- Have you accepted the $100k offer for 300 users? (we haven’t yet in my company)
2- Have you already done some sort of a business case to access the value? (we initiated one in my company)
Yes, we have accepted. We wanted to have the opportunity to test Copilot with a broader persona of users across the enterprise to better understand it’s value. I see value in potentially two main categories – time savings and quality of content with the former being the predominant one. However, as Tony correctly pointed out in this article, it’s one thing to quantify time savings and another to equate that to impact on the company’s bottom line.
A large part of our proof of concept will focus on what (if any) high value use cases exist and to which persona they apply.
hello Nick, totally agree with you.
the hard part will be measure it 🙂
there are some cases where you can actually measure immediately if the copilot helped, but in general it is very hard
Hi Nick,
I’d love to know what kind of ROI your organization observed during your POC experience. Would you be willing to share your feedback?
Hi Tony!
Thanks for this insightful article.
My company is being offered to join M365 Copilot Early Access Program. We would have to pay $100k to run the test for 300 users… We have not accepted yet…
Do you know if it is the same price / number of test users for each of the 600 companies?
We are not keen to pay that much to run a test / be guinea pigs.. anyone having being in the same situation?
I suspect that the cost is similar for all customers in the trial. Of course, it might be discounted by the local sales team in some cases. Microsoft wants the customers who participate to prove their “dedication” by committing people and financial resources to testing Copilot in a customer environment, and this is one way of showing that commitment. Maybe you can have a chat with your sales representative and see what they can do. Or just wait. Who wants to pay that kind of money to test pre-production software?
Hi Tony! Thanks for such an informative and interesting article. I work for a company called PPT Group – a team of business consultants who focus on digital transformation and technology strategy within businesses. One of our clients has been selected by Microsoft as one of the 600 customers for the Copilot Early Access Programme and we will be assisting them with understanding where Copilot can increase productivity in their environment & what the correct areas are for implementation. It would be great if we could potentially have a one-on-one discussion with you. Let me know if you’d be interested?
I’ll be at TEC in Atlanta (Sept 19-20) if that helps…
We’re based in South Africa so the TEC might be a bit out of our reach but thank you for letting us know. Perhaps we can still make arrangements to attend! In the meantime, would you be willing to entertain a Zoom call with us?
You should come to TEC. It’s a great two-day event where you can meet with lots of industry personalities. https://www.quest.com/the-experts-conference/
I was in SA last month… but anyway, I can do a call with you if you like. DM me on Twitter.
How do you decide, who gets to be more productive in your organization and who should be left behind and made redundant?
Why is intelligence a commodity in the first place?
Why is it not a basic universal human right just like food, health and education? Who gets to decide that?
When we have used collective knowledge of an entire civilization to train the model itself, without any royalty in first place.
Organizations always make choices about what licenses they assign to people. Someone might get Office 365 F1 if they only need email and calendar access. Others might get Office 365 E5 if they need access to some of the extended compliance features. Copilot is just another variant of the game.
Very timely article. Do you know if there is a ‘minimum number of licenses’ requirement?
I don’t.
Nice article. Conversations like this are already happening in my company and for me I go through this mental checklist of people in the organisation and think…could their role benefit from copilot? Highly likely a copilot pilot will kick off.
Hi Tony, Excellent piece. I think the statement “Not everyone needs a Microsoft 365 E5 license” should be caveated a little more. ‘Not everyone needs a Microsoft 365 E5 license unless they are going to utilize the available features’.
True! Perhaps the exercise to assess people for Copilot licenses will turn up situations where some are revealed not to need even the base license they have.
This caught me by surprise
“ a lawyer finishes a case submission a half-hour faster because Copilot helped them to craft the text and is able to bill another customer $500 for that half-hour”
I hope that this can be widely accepted as unethical if not illegal. The time it takes to do an hour of work will always be precisely one hour. The quantity and quality, and even the cost is flexible but time is not.
Unhappily, legal overcharging does occur (here’s the best example at present: https://www.legaldive.com/news/Twitter-sues-Wachtell-90M-success-fee-X-Corp-musk-edgett-gadde/686020/).
It’s common in some places for lawyers to assign a fixed charge for doing something (like conveyancing of a house). If Copilot helps them get a job done sooner, they can do other chargeable work in the time saved.
Leaving aside the ethics question, if that same Legal Eagle uses Copilot to complete an unbillable regulatory filing 30 minutes faster with Copilot and spends that time on a task that is billable….
The world of AI opens up new possibilities…
Hi Tony, nice written. Just one thing, M365 Business Standard already includes Office desktop apps. Did you mean M365 Business Basic? These are without Office apps.
Brain fart. Thanks for pointing it out. Fixed!