At Ignite this year, Microsoft announced the release of the Office 365 ATP Recommended Configuration Analyzer, otherwise known as ‘ORCA’.
Office 365 Advanced Threat Protection is an add-on service for Office 365 to protect users against malicious threats by email, URL and collaboration tools. As this is an advanced solution, your customers who have recently purchased the E5 licenses or ATP Plan 1 or 2 may struggle with the configuration of these features.
In this article, we will walk through the setup and commands used to run this report on your tenant and review suggested recommendations from Microsoft.
Office 365 ATP: Plan 1 and Plan 2
There are two different plans which are GA, ‘Plan 1’ and ‘Plan 2’. Below I’ve provided a table from Office 365 ATP Plans which summarizes what’s included in each plan.
Advanced Threat Protection Plan 2 is included in the Office 365 Enterprise E5, Office 365 Education A5, and Microsoft 365 Enterprise E5. ATP Plan 1 is included in Microsoft 365 Business.
ORCA Installation
As previously mentioned, ORCA is an advanced solution, so as the Admin you’ll have to execute these for your users. To get started, follow these steps:
- Launch PowerShell as an administrator
- Type“install-module -name orca” and press Enter
- Login to Exchange Online with the Exchange Online PowerShell Module. You can find the connection instructions here.
- Once logged in you can execute the command Get-ORCAReport and your report will be generated
- Save this as an HTML file and then open in your browser
When ORCA is creating the report, it extracts settings from Exchange Online Protection (EOP), anti-phishing and anti-spoofing policies, along with Safe Links and Safe Attachment configurations and then compares them with Microsoft best practice recommendations. You can then decide what you want to change or implement after.
ORCA Report
Here is an example report from ORCA, the first item shows the current number of Recommendations and OK items.
A summary of each area is provided in the latest 1.2.1 version for each category.
Let’s check out the Content Filter Policies. Our bulk level is currently set to 7 however Microsoft recommends setting this to 6 or lower. What I like about this report is Microsoft has also added direct links to these areas to learn and get more information about each policy and setting.
Next, we’ll review the IP allow list example.
One of the Recommendations is to remove IP address from IP allow list. However, this is dependent on how your email routes inbound to your Office 365 tenant. If you use a third-party mail filtering service, then this setting would be acceptable because your IP allow list would be managed here. If your first point of inbound mail is Office 365 then you would keep this setting vacant as Microsoft advises to avoid duplicate content filtering.
Conclusion
These are just a few examples from the ORCA report. Microsoft has done an outstanding job of creating a best practice analyzer for your Office 365 ATP configuration. A lot of administrators may not know about these features or what is added with ATP and the added benefit in the protection of your tenant. I highly endorse running the ORCA report and taking the time to evaluate the recommendations provided by Microsoft. If the recommendations make sense to implement, ensure you follow a good change process and carry out testing in a test tenant environment. Microsoft makes it easier to get your tenant configured correctly with ORCA and I feel it will help you in configuring your settings correctly. This will lead to preventing and protecting against malicious content directed toward your tenant.
Please use PowerShell ISE to run Get-ORCAReport.,
I got bunch of errors multiple time using regular powershell
Use the latest version: https://www.powershellgallery.com/packages/ORCA/1.10.6
Hi,
when i run the Get-ORCAReport command everything goes well till the last line and at the end it comes like this,
01/27/2021 12:46:10 Generating Output
01/27/2021 12:46:10 Output – HTML
Invoke-Expression : The term ‘C:\Users\Mohamed’ is not recognized as the name of a cmdlet, function, script file, or
operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try
again.
At C:\Program Files\WindowsPowerShell\Modules\ORCA\1.9.11\Outputs\output-html.ps1:571 char:13
+ Invoke-Expression $OutputFile
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (C:\Users\Mohamed:String) [Invoke-Expression], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException,Microsoft.PowerShell.Commands.InvokeExpressionCommand
01/27/2021 12:46:12 Complete! Output is in
PS C:\>
It dosn’t export the report or save it anywhere. Please help.
Thank you.
Microsoft has since implemented Orca in to the security center. If you login as your admin account and go to Security – Threat Management – Policy – Configuration Analyzer
This is in Preview so it may or may not be in your tenant.
https://docs.microsoft.com/en-us/microsoft-365/security/office-365-security/configuration-analyzer-for-security-policies?view=o365-worldwide
Hi Tony,
Thanks for the reply. i can see the recommendations in the security center. but i don’t see an option to export it. when i use powershell to export it, i get the error message in the comment above.
Hi,
when i run the Get-ORCAReport command everything goes well till the last line and at the end it comes like this,
01/27/2021 12:46:10 Generating Output
01/27/2021 12:46:10 Output – HTML
Invoke-Expression : The term ‘C:\Users\Mohamed’ is not recognized as the name of a cmdlet, function, script file, or
operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try
again.
At C:\Program Files\WindowsPowerShell\Modules\ORCA\1.9.11\Outputs\output-html.ps1:571 char:13
+ Invoke-Expression $OutputFile
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (C:\Users\Mohamed:String) [Invoke-Expression], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException,Microsoft.PowerShell.Commands.InvokeExpressionCommand
01/27/2021 12:46:12 Complete! Output is in
PS C:\>
It dosn’t export the report or save it anywhere. Please help. Thank you.