In an email from a reader I was asked about Office 365 Secure Score and whether it will help their organization to know when hackers have gained access to their Office 365 data. The short answer is no, that’s not the purpose of Secure Score.
Office 365 Secure Score analyzes your tenant and provides a report on which security features of Office 365 you’ve have or haven’t implemented. Secure Score then provides access to either a direct remediation (e.g. click here to change this), or guidance for how to manually implement a recommended security feature. The “score” that Secure Score provides is simply a way for you to benchmark your tenant security over a period of time.
What Secure Score won’t do is alert you to intrusions, such as a compromised password being used to access data from an unusual location. For that type of activity-based detection and alerting in your tenant Microsoft provides Office 365 Advanced Security Management (ASM), which is available in the E5 license.
Here’s an example of ASM in action. I enabled it some time ago for my own tenant, and then during October and over the New Year period I was travelling overseas for conferences and a vacation. While I was travelling I triggered some “Impossible travel activity” alerts in ASM.
The “Impossible travel activity” was due to me connecting my Freedome VPN service to their Australian point of presence, and then accessing an Office 365 application through that VPN connection. In reality, I’d probably left Outlook running while I connected the VPN to circumvent some Australian geo-blocking. To ASM this looks supicious, because I obviously can’t be in two countries at once, and believe me it takes more than 313 minutes to cross the Pacific. ASM considers this a “Low” severity alert, because they’re smart enough to know that such things as VPNs exist. That’s not to say that the same activity wouldn’t be considered a critical alert to another organization, perhaps a breach of password policy if someone has shared their credentials with another person, and you can certainly customize ASM policies to align with your organization’s view of such things.
Another nice feature of ASM is the activity log, providing a view of activity for specific accounts such as the service account I have set up in my tenant for Cogmotive reporting. If you’re ever investigating someone’s login activity, or need to keep an eye on usage of powerful service accounts, this activity log is going to be very useful.
So in summary, Secure Score is going to help you to improve the security posture of your Office 365 tenant, but Advanced Security Management is what you would use to monitor and alert you to suspicious activity.
