Block Copilot Access to Individual Office Documents

New Sensitivity Label Setting Prevents Transmission of Data to Microsoft Content Services

Updated 8 Nov 2024

In March 2024, Microsoft announced Restricted SharePoint Search. Now in public preview, Microsoft says that Restricted SharePoint Search gives organizations “time to review and audit site permissions” and “maintain momentum with your Copilot deployment while you implement robust data security solutions.” The product documentation highlights the need to stop Copilot from accessing content in “sites that haven’t undergone access permission review or Access Control Lists (ACL) hygiene, and doesn’t have data governance applied.” In other words, Copilot for Microsoft 365 can extract and use information from sites where access control might not be what it should be and reuse that information in the responses it generates for user prompts.

Restricting SharePoint search to an allowed list of 100 “curated” sites might stop Copilot. It also stops humans from finding information stored in sites that aren’t on the list. To me, handicapping search in this way represents the sorry legacy of previous Microsoft collaboration strategies. In general, Microsoft 365 tenants simply have too many SharePoint sites (and Teams) with information spread far and wide across the sites in an uncoordinated and uncontrolled manner.

Blocking Content Analysis

A new control over the access of Copilot for Microsoft 365 to the information stored in Microsoft 365 is described in message center notification MC802004 (15 June 2024, Microsoft 365 roadmap item 398991). This is a preview capability imposed through sensitivity label settings to prevent applications from sending information from Office documents and emails to Microsoft content services. Items protected by sensitivity labels with double key encryption already block access by services like Copilot because they don’t have access to the customer key that’s necessary to decrypt content.

Sensitivity labels have advanced label settings, some of which are not exposed through the GUI when configuring labels through the Purview compliance portal. Like other advanced label settings, the new BlockContentAnalysisServices setting can only be managed using PowerShell. For example, this code connects to Exchange Online and to the compliance endpoint before running the Set-Label cmdlet to apply the new setting:

Connect-ExchangeOnline
Connect-IPPSSession

Set-Label -Identity "Market Sensitive" -AdvancedSettings @{Tooltip="This label stops a file being referenced by Microsoft 365 Copilot";BlockContentAnalysisServices="True"}

Obviously, applying a block through a sensitivity label targets individual documents and is more precise than the rather crude mechanism applied by Restricted SharePoint Search. The block only applies to files labeled after adding the advanced setting. To check that the block is in place, you can run a simple command (below) to check the value of the blockcontentanalysisservices setting, or use the sensitivity label settings report script.

(Get-Label -Identity 'Market Sensitive').Settings

[tooltip, Information that if it leaked out could compromise our business]
[displayname, Market Sensitive]
[contenttype, File, Email]
[color, #008b8b]
[blockcontentanalysisservices, True]

Blocking Access for Copilot

When a sensitivity label that blocks content services is present for a document, the Copilot options in Office apps are disabled (Figure 1). This happens because the use of Copilot features like summarizing the text in a Word document or analyzing data in an Excel worksheet requires information to be transmitted to the LLMs used by Copilot.

When working in documents that don’t block access to content services, a user can explicitly reference the blocked document in a prompt to allow Copilot to access its content. As an example, we’ll use the source document for this article, which has the Market Sensitive label updated earlier to block access to content services.

Figure 2 shows a prompt to ask Copilot for Word to draft a note about blocking access to content services with sensitivity labels. This article is referenced, so the prompt can use its content even though the file has the Market Sensitive label.

Figure 3 shows the results generated by Copilot for Word. The left-hand side is the document created without referencing this article. The text comes from internet sources and other documents that Copilot can access within my tenant. The generated text is generic and incorrect in part, which is the expected result for an unfocused and imprecise prompt. The right-hand document is the result of explicitly including this article in the prompt. The accuracy of the generated text is much better. You can also see that the output document inherits the Market Sensitive label from the article because it was referenced by the prompt.

These results show that applying a sensitivity label with the block content analysis services setting is effective at blocking Copilot access to individual files. Unless the user makes an explicit decision to allow Copilot to access a labeled document’s content, it will be ignored.

Be aware that the sensitivity label does not block searches of document content. If sensitive information is found in file metadata (like the document name), or a match is made against the semantic index, a document can be listed but not opened by Microsoft 365 Chat (BizChat). For example, Figure 4 shows how Microsoft 365 Chat can find a document with a label blocking content services using the document’s title. However, the label prevents Copilot from accessing the content.

The Downside of Blocking Content Services

The downside, as noted in Microsoft documentation, is that when content is protected by a label that blocks access to content analysis, “some services won’t work as designed, such as data loss prevention policy tips for Outlook, automatic and recommended labeling, and Microsoft Copilot for Microsoft 365.” The features that don’t work include suggested replies in Outlook and text predictions in Outlook and Word.

DLP and automatic labeling don’t work because applications note the block imposed by the label setting and don’t send content for processing by the service, meaning that policies cannot decide if files meet their criteria. For instance, DLP policy tips that usually signal the presence of sensitive information types in messages won’t appear because Outlook won’t send message content to the server. Take the example shown in Figure 5. The left-hand message has the Public label, and a policy tip is visible to advise that credit card information is detected. The right-hand message uses the Market Sensitive label which blocks transmission to content services. DLP cannot detect the credit card information and no policy tip appears.

Losing a feature like policy tips is a small inconvenience when measured against the ability to control highly sensitive information. In any case, service-based processing for DLP policies will block sharing even if Outlook can’t detect a possible violation.

Access for Copilot for Microsoft 365 in Non-Office Scenarios

Sensitivity labels that block sending content to Microsoft only currently affect the Office apps (which is why you need to run a specific version of the Microsoft 365 apps for enterprise), The public preview starts with the Current Channel Preview V2406 in early June. Microsoft expects that General availability will occur worldwide via the Current Channel release in July.

Copilot access to information is not blocked in other scenarios. For instance, the Microsoft 365 Copilot chat app can find and use Office documents even when they have a sensitivity label that blocks access (Figure 6).

Control Slowly Coming

Microsoft announced Copilot for Microsoft 365 in March 2023. Fifteen months later, it’s reasonable to ask why the controls to limit Copilot access to sensitive information are still evolving. After all, Microsoft surely realized the problems that can occur when AI tools have untrammeled access to information in SharePoint Online sites as they worked on the initial development of Copilot, including the preview trials with selected customers.

Although it’s frustrating to see the slow appearance of controls, I think it’s fair to say that it reflects the complexity of an environment where multiple moving parts need to work together to ensure that limits are respected. It certainly underlines the need for organizations contemplating the use of AI to pay attention to information governance and think about how they manage Office documents and other files in SharePoint Online.