If you have spent any time hardening a Microsoft 365 tenant, you have run into ORCA. It is one of the few genuinely free, Microsoft-maintained tools that reviews your tenant configuration and tells you where it deviates from Microsoft's own recommended settings.
But ORCA is also widely misunderstood. People treat it as "the Microsoft 365 security audit," when it is really one lens on one part of the tenant. This post explains what ORCA actually is, what it checks well, and where it stops — so you know when it is enough and when you need more.
What ORCA is
ORCA — the Office 365 Recommended Configuration Analyzer — is a free tool from Microsoft that reads your Microsoft 365 and Exchange Online configuration and compares it against a recommended baseline. It produces a report that highlights settings that differ from what Microsoft considers secure or recommended.
It is read-only. It does not change anything. It gives you a snapshot of configuration drift against Microsoft's guidance.
What ORCA checks well
ORCA is at its strongest on Exchange Online and tenant-wide mail settings. Its checks tend to cover things like:
- Transport and mail-flow configuration
- Anti-spam and anti-phishing defaults
- Client access and legacy protocol settings
- Mailbox and tenant-level configuration that affects mail security
- Settings that Microsoft has flagged as recommended in its own documentation
For an Exchange-heavy organization, ORCA surfaces real, actionable gaps — especially around mail flow and the settings that are easy to leave at insecure defaults.
What ORCA does not check
This is where people get into trouble. ORCA is not a complete Microsoft 365 security audit. It is not designed to be. Specifically:
- It is Exchange-centric. It does not give you deep coverage of the Entra ID identity surface — Conditional Access policy quality, MFA registration coverage, privileged role sprawl, guest access governance, or sign-in risk.
- It is point-in-time. You run it, you get a report. It does not monitor the tenant over time, and it does not tell you what changed since last quarter.
- It does not produce a client-ready deliverable. The output is a technical report. If you are an MSP handing something to a client, you still have to translate it.
- It does not map across frameworks. You get Microsoft's recommendations, not a crosswalk to CIS, CISA SCUBA, EIDSCA, or the compliance controls your auditor is measuring you against.
In other words, ORCA tells you a lot about Exchange and relatively little about identity — and identity is where most modern Microsoft 365 attacks land.
How to run ORCA
ORCA runs as a script against your tenant with read-only permissions. You point it at the tenant, it reads the configuration, and it produces a report you can review. For a single Exchange-focused review, it is quick and useful.
The catch is operational: you have to run it, read it, and remember to run it again. Nothing about ORCA turns a one-time check into an ongoing process.
Where a full audit picks up
A complete Microsoft 365 security audit covers the surfaces ORCA does not — and runs the frameworks ORCA is not mapped to. That means:
- Identity first: Conditional Access coverage and gaps, MFA enforcement, legacy authentication, privileged role assignments, guest access, sign-in risk.
- Beyond Exchange: Teams external and guest access, SharePoint and OneDrive external sharing and link defaults, Defender onboarding and policies, audit logging.
- Mapped to benchmarks: CIS Microsoft 365 Foundations, CISA SCUBA, EIDSCA, ORCA, and community best practices — so every finding carries the control it maps to.
- A real deliverable: a severity-ranked, remediation-ready PDF you can hand to a client or drop into an audit file.
You can see exactly what that looks like in our sample report, or read how each framework fits together in the benchmark guides.
The short version
Use ORCA. It is free, it is Microsoft's, and it catches real Exchange configuration drift. Just do not mistake it for the whole job. ORCA is one lens; a Microsoft 365 security audit is the full picture — identity, collaboration, devices, and the frameworks your auditor actually measures you against.
If you want the full picture without running five tools by hand, run an audit against your tenant and get the report in minutes.