Microsoft has simultaneously been tight-lipped and very vocal about Microsoft 365 Copilot, creating a lot of buzz in the run-up to its GA (General Availability) on November 1^st;  but keeping access to Microsoft 365 Copilot behind a pay-to-play Early Access Program. Like many private previews, customers are kept under NDA, and can’t talk openly about how they are using it within their own organization.

The fact Microsoft is at GA so soon appears impressive. If you consider Intelligent Recap for Teams Meetings was demonstrated back in 2018 and arrived in tenants five years later, Microsoft is doing extremely well to have paying customers onboarded to Microsoft 365 Copilot already. But is Microsoft jumping the gun by releasing it to GA?

Copilot Makes an Outing at Microsoft Envision

Less than two weeks before GA, Microsoft was ready to show Microsoft 365 Copilot to the general public.

At Microsoft Envision the Tour London, the theme for the day was AI; and in Microsoft’s Showcase section, Microsoft 365 Copilot was available to demo on every Modern Work stand; alongside Microsoft staffers willing to answer almost any questions folks had.

At the conference, I was working on a stand in the Microsoft 365 Copilot showcase area. It made for an excellent opportunity to see if Copilot lives up to the hype and share that insight with you.

On First Impression: Was it Worth the Money?

On Practical 365, Tony has covered Microsoft 365 Copilot costs and the need for Microsoft 365 licenses; which is a step up for a certain segment of mid-market and enterprise customers.

I’m not going to argue the necessity of Microsoft 365 licensing vs Office 365 licensing for Copilot to function; I know that customers getting ready for Copilot do feel they need good information governance, and in particular, they want classification and protection of sensitive documents.

For the Curators and Those Who Mold and Shape Information: Absolutely

If you select the right people to provide Copilot licensing to, and then support the adoption properly, and as Tony said a few months ago – begin with a Pilot and measure results, then it absolutely does provide value for the money.

Tony is correct in his assertion that folks who create customer-facing material like proposals or write reports based on analysis of information are the right people to start with. As he mentions, Microsoft says it’s best used by someone who works with an abundance of information in Microsoft 365. A massively important point to this is that those folk will find Copilot most effective when they know where the information is. Copilot isn’t, as tested and according to Microsoft folks sharing in public, going off and wildly bringing in random information as some people have presumed. In the context of using it in apps like Word, it is more likely to use information from the underlying GPT-4 LLM than bring in information from other documents, and when referencing documents, its starting point appears to be the folder you are in.

The real-world application of this to roles like Tony mentioned, such as a bids manager, is that they’ll have a way of working that is ripe for efficiency savings, but is functional: they are likely to store customer requirements documents alongside the proposal they are creating, and they often have a location or repository of content or previous bids they’ll use. The observed behavior of Copilot appears to work in a way that works efficiently in this context – if it brought in data from further afield then I suspect the user would lose confidence pretty quickly; they expect to stay in control, as after all – it’s a Copilot.

Examples like this are where the time savings will be clear. The types of roles where it provides the most value are likely to require skilled people in short supply, and the people in those roles are already busy. The effect should be to multiply their productivity and allow them to improve the quality of their output. You would expect that they are already efficient and already competent; the bottom line savings may be that they are overworked and without a Microsoft 365 Copilot license or an extra head in the team, quality will be suffering.

Naturally, that isn’t going to be everyone in your organization. As I’ve said before on the Practical 365 podcast – if you have a productivity problem already, Copilot isn’t likely to solve it.

As an aside; I’ve heard Microsoft say a few times that they feel Microsoft 365 Copilot will give a leg-up to people early in their careers. I’m not so sure; tenant data issues aside, from what I’ve seen, people benefit most from Copilot if they already know what the output should look like and have the experience to know if the analysis of the information doesn’t look quite right.

Think of Copilot in apps like Word, PowerPoint, and Excel like a calculator: If you already know how to get the result manually, it’s a massive timesaver. If you don’t, it may be dangerous because you won’t know if the resulting outputs generated along the way are correct.

Outlook, Teams, and Microsoft 365 Chat: Too Early to Say

Microsoft was keen to show off what is undoubtedly a fantastic set of features that fundamentally serve one purpose: to save time catching up with stuff.

Value from that is hard to measure. Was the person using Outlook and Teams effectively to begin with? Could encouraging people to @ mention people in email and Teams actually save more time – so that you don’t need to read even the summary of what’s going on in a Teams channel or email chain – and just skip to the bit where you are asked a question?

It’s hard to argue that the email summarization doesn’t work. As you can see below, in Microsoft’s demo tenants, it works pretty well (figure 2). In email, it highlights who said what, and does so on a thread-by-thread basis simply and effectively.

In Teams channels, the value may come from busy channels; but in Microsoft’s demo environment below, I’d argue that it would have taken less time to read the channel messages than asking Copilot to do so, and then waiting a little while it does it’s thing, and reads through it’s summary (figure 3).

For Meetings, there is a bit of a crossover where Intelligent Recap and Microsoft 365 Copilot (figure 4) serve superficially similar purposes.

What Copilot offers appears to be in-context meeting summarisation; if for example, you are late to a meeting; or you are pulled out of a meeting in progress for five minutes, then you can ask what happened while you were away. Or, if you missed the meeting you can dig into the details; and post-meeting is where the crossover begins.

Copilot functionality in this context feels like an added bonus that is additive to Intelligent Recap. If Microsoft doesn’t include Intelligent Recap with Microsoft 365 Copilot then I suspect customers will feel it is lacking compared to elsewhere in the suite.

Feedback from customers I spoke to regarding both these features for Teams has been varied – from very positive to a little lackluster; especially from organizations that balk at the thought of recording meetings as well as transcribing them to make use of Intelligent Recap.

Generally, though, Microsoft 365 Copilot is likely to drive Teams Premium uptake as a general time-saver for meetings at a much lower cost with a lower bar to entry. It is a clear win for people organizing and taking the action from many meetings, like project managers, for example. I’ll cover Intelligent Recap in more detail in a future article, though.

At Envision, one aspect I didn’t get to see enough of to comment on was the Microsoft 365 Chat. Yes, it exists and can summarise things. The devil is in the details; for example, customers of mine who have been using Bing Enterprise Chat believe it’s not as capable as ChatGPT and produces worse results.

Even with Microsoft 365 Chat having access to enterprise data and the ability to search on the web doesn’t automatically mean it will be seen by users as better than ChatGPT (or an Azure OpenAI deployment of GPT-4 or GPT-4-32k with a suitable system prompt and chat front end).

Understanding how well received it will be, will depend on how widely it finds data in comparison to how the Office apps appear to find data in the context of the open documents; what it can do with that data – and because it is the place most likely to be seen as effectively an enterprise “ChatGPT”, what controls and monitoring are available for it.

Does it Unlock the Full Power of PowerPoint? Not yet, but it Probably Won’t Need To.

One of the underlying promises of Microsoft 365 Copilot is that it will bring the ability to unlock the full feature set of Microsoft 365 Apps (Office) to users; and in comparison to what was announced, it does not.

It can, just about, deliver on the main headline feature: take a document and create a PowerPoint deck from it.

What I believe we’re seeing with Microsoft 365 Copilot in Figure 5, above, is that it’s capable of taking a document, and then generating several slides of summarised content. Then for each slide, it executes PowerPoint Designer against the slide, selecting the first option. To validate my theory, I made a quick test app in Python, using a deployment of GPT-4 in Azure and saw similar results.

It can do more than that of course; for example, you can ask it to insert appropriate pictures into slides. But, as you’ll see above if you look closely – sometimes it answers in a similar way to the Windows Copilot preview – with instructions on how to do the task, rather than getting stuck in and doing it for you.

I have to point out though – I’ve only seen the web apps in public – so it could be different in the full-fat PowerPoint app. And, at a basic level, it’s not bad; and for people who don’t use many of the newer features, like Designer and stock photos, it is going to impress.

But if you were hoping for something reminiscent of what Microsoft originally demonstrated back in March, as shown in Figure 6 below, then you will be a little disappointed.

On balance, though, I would recommend not judging Microsoft too harshly on Copilot in PowerPoint.

In the real world, most people in enterprises use corporate templates that must be adhered to; Designer can work well with properly formatted templates, so it’s a reasonable assumption that Copilot will do so too.

And if the choice was to wait another six months for PowerPoint to have some rad AutoGPT style generative design capabilities, or to get your hands on Microsoft 365 Copilot now, you wouldn’t choose to wait; the rest of the capabilities elsewhere more than makeup for this.

A Strong Start for Microsoft 365 Copilot, But Time Will Tell

Overall it’s hard not to be impressed. If you’ve listened to the Practical 365 podcast you will know that I’m sometimes disappointed by Microsoft not delivering products when they say they will. Given the demand for AI from business execs, Microsoft clearly knew that failure to deliver wasn’t an option.

And despite missing some features in the version Microsoft publicly made available for demonstration at Envision – it is delivering well on core tasks that will drive the business cases to buy it.