How to use the Azure Content Moderator in Office 365

Office 365 and SharePoint Online both offer social functionality that includes the use of chats and group conversations. However, if you want to moderate these discussions, there aren’t any tools available to do so right now. A solution for this is programming. We can implement the Azure Content Moderator Service inside SharePoint and Teams to create moderated debates, detect inappropriate and undesirable content and filter it if necessary.

In light of current affairs regarding content moderation with Facebook, Twitter, and other social networks, it is more important than ever for enterprises to screen the content that is distributed by and to their users. Until recently, businesses relied on human moderators to look at the generated content or counted on their customers to alert them of unwanted content. However, because of the monetary costs involved, and the risk of devaluing the organization’s brand, this approach is becoming increasingly unpopular.

The Azure Content Moderator is one of the Azure Cognitive Services that checks the text, image, and video for content that could be harmful, offensive, or inappropriate. When this content is found, the service applies flags which alerts human moderators, or it correct this automatically.

The Moderator consists of two main parts: a set of API’s (based on REST and a .NET or Python client SDK) that can be used to receive the content to moderate, analyze it and send back the moderation results, and a Human review tool that offers an interface to gather the content, send it to the API in an automated way, and show to a human moderators the potential problems. The tool also offers ways to create simple flows and custom tags to facilitate the review of a moderator.

In this article, I will demonstrate how to create a basic Moderated SharePoint Discussion Board using the Azure Content Moderator API. Although the classic SharePoint Discussion Board List is not (yet) converted to a “Modern” SharePoint interface, it offers the infrastructure to create a moderated list, that, using a SPFx interface, can be easily integrated in any SharePoint page. Unfortunately, in Microsoft Teams, the Conversation tab doesn’t have an API that allows us to intercept messages and modify them. However, if you use a SharePoint Discussion Board, plus a custom SPFx interface, it is easy to integrate this into Teams. On the other hand, Yammer has a REST interface that could eventually be used to moderate a discussion in a similar way as with the SharePoint Discussion Board.

Creation of the Azure Content Moderator Service

To use this service, you need to create the Content Moderator service first. You can do this from the Azure control page, sign in with your administrator’s account and create a new Resource Group or you can use an existing one. In the Group add a Cognitive Services resource, giving it a name, Azure Subscription, and Location. Currently, there is only one pricing tier. The Location determinates the Endpoint of the service as you can see in the Overview tab when the service is created; save this value for later. Go to the Keys section and save the value of KEY 1 as well.

Interaction with SharePoint

As previously mentioned, we will use a Discussion Board List in SharePoint. Firstly, create the List in one of the Site Collections in the tenant. To intercept the creation of a new thread in the List, the best option is to use a WebHook implemented as an Azure Function. You can find complete instructions about how to create and subscribe a Function as a SharePoint WebHook can be found in the Microsoft site here. In the example developed by Microsoft, a new item created in the List is added as a queue in a storage account to process it in an asynchronous way. In our case, we can act directly gathering the Body of the created item and calling a routine DoModeration that makes the call to the Azure Service:

static void DoModeration(ClientContext SPClientContext, ListItem ItemChanged)
 {
     string disBody = string.Empty;
 // Read the post Body disBody = ItemChanged["Body"].ToString(); // Make a stream of the body text byte[] disBodyByteArray = System.Text.Encoding.UTF8.GetBytes(disBody); MemoryStream disBodyStream = new MemoryStream(disBodyByteArray); // Get the Moderation from Azure and modify the Body AzureContentModerationResults myModeration = CheckContentForModeration(disBodyStream); string myModifiedBody = ModifyText(myModeration.AutoCorrectedText, myModeration); // Send back the modified Body to SharePoint ItemChanged["Body"] = myModifiedBody; ItemChanged.Update(); SPClientContext.ExecuteQuery();
 }

This routine receives the SharePoint context and a reference to the new List Item. First, the routine extracts the text in the Body, convert it to a byte array stream and send it to the routine CheckContentForModeration for moderation. The routine returns an object of the type AzureContentModerationResults containing the results from the Azure Service and sends the AutoCorrectedText to the ModifyText routine. Finally, the modified string is pushed back to the Body of the List Item.

Note: the complete WebHook code can be found in the GitHub report.

The call to the Moderator Service is done by the CheckContentForModeration routine, using the Endpoint and Key found earlier:

static AzureContentModerationResults CheckContentForModeration(MemoryStream TextToModerate)
 {
     string AzureContModBaseURL = "https://westeurope.api.cognitive.microsoft.com";
     string ContModSubscriptionKey = "dabb15da19a14baea06a71bfxxxxxxxx";
 ContentModeratorClient contModClient = new ContentModeratorClient(new ApiKeyServiceClientCredentials(ContModSubscriptionKey)); contModClient.Endpoint = AzureContModBaseURL; var objResult = contModClient.TextModeration.ScreenText("text/plain", TextToModerate, "eng", true, true, null, true); string jsonResult = JsonConvert.SerializeObject(objResult, Formatting.Indented); return JsonConvert.DeserializeObject<AzureContentModerationResults>(jsonResult);
 }

The AzureContentModerationResults object has the form:

public class AzureContentModerationResults
 {
     public string OriginalText { get; set; }
     public string NormalizedText { get; set; }
     public string AutoCorrectedText { get; set; }
     public object Misrepresentation { get; set; }
     public Classification Classification { get; set; }
     public Status Status { get; set; }
     public PII PII { get; set; }
     public string Language { get; set; }
     public Term[] Terms { get; set; }
     public string TrackingId { get; set; }
 }

The Moderator Service recognize principally two types of information in the text to analyze:

1. Profanity words in more than 100 languages and custom blacklists. The detected words are in the array of Term objects, where you can find also the position of the word in the complete text context.
2. Personally Identifiable Information (PII), that includes email addresses in any language, and social security numbers, phone numbers and addresses for the USA and the UK only. If any PII is identified, a modified text is returned masking it with spaces to make it difficult to read by automated processes.

The ModifyText is a very basic routine that uses the PII modified text, and replace the found Terms with asterisks:

static string ModifyText(string InputString, AzureContentModerationResults ModerationResult)
 {
     foreach (Term oneWord in ModerationResult.Terms)
     {
         string termModified = oneWord.term.Remove(1);
         for (int charCount = 1; charCount < oneWord.term.Length; charCount++)
         {
             termModified += "*";
         }
         InputString = InputString.Replace(oneWord.term, termModified);
     }
 return InputString;
 }

When everything is compiled and running, if a user creates a new thread in the Discussion Board:

The WebHook will send the text to the Moderator Service, modify it and the final text will be shown (note the modifications in the email address and the detected terms):

Conclusion

To conclude, we can ensure that the Azure Content Moderation Service can be used to fulfill the increasing necessity of editorial control for social media content. In the article, we saw its application for text, but the service can also be used for moderation of pictures and videos.