Ask five Microsoft 365 admins how they audit a tenant and you will get five answers: ORCA, EIDSCA, a Secure Score screen-share, a spreadsheet, or "we pay a consultant." Each has a place. None is complete on its own.
This post compares the three most common free approaches — ORCA, EIDSCA, and manual Secure Score reviews — and explains which to use when.
ORCA: Exchange configuration
ORCA (Office 365 Recommended Configuration Analyzer) is Microsoft's free analyzer for Exchange Online and tenant mail configuration. It compares your settings against Microsoft's recommended baseline and reports deviations.
- Best for: Exchange-heavy environments, mail-flow and anti-phishing review, catching mail configuration drift.
- Weak at: identity. ORCA is not built to assess Conditional Access quality, MFA coverage, or privileged role sprawl.
- Shape: run it, get a technical report.
EIDSCA: Entra ID configuration
EIDSCA (Entra ID Security Config Analyzer) is an open-source baseline for Entra ID. It checks the identity configuration — authentication methods, Conditional Access, consent, guest defaults, privileged roles — and returns findings mapped to control IDs.
- Best for: assessing the identity layer against a real baseline. This is where most modern attacks land, so it is where you should spend the most time.
- Weak at: everything that is not Entra ID — Exchange, Teams, SharePoint, Defender.
- Shape: assess against controls, get findings.
Secure Score: a metric, not an audit
Microsoft Secure Score is Microsoft's aggregated measure of your tenant's security configuration, surfaced in the Defender portal. It is genuinely useful as a headline indicator and a source of prioritized recommendations.
But a manual Secure Score review has real limits:
- It is a score, not a control-by-control assessment. It tells you roughly where you are, not exactly what is wrong and why.
- Its recommendations are Microsoft's, not a framework's. It does not map to CIS, CISA SCUBA, or EIDSCA control IDs.
- It is relative. Two tenants can have the same score with different weaknesses.
- It does not produce a deliverable you can hand to a client or auditor.
- It is point-in-time and drifts.
We go deeper on this in Secure Score is not a security audit.
So which should you use?
The honest answer: all of them, together. They cover different surfaces, and a tenant is only as strong as its weakest one.
| Approach | Covers | Misses | Output | | --- | --- | --- | --- | | ORCA | Exchange / mail config | Identity, Teams, SharePoint | Technical report | | EIDSCA | Entra ID config | Exchange, Teams, SharePoint | Findings by control | | Secure Score | Aggregated posture | Control-level detail, evidence | A number | | Full audit | All surfaces + all benchmarks | — | Client-ready PDF |
If you can only run one thing, run identity (EIDSCA) — that is where breaches start. But the moment you need to prove anything to a client or an auditor, you need coverage across every surface and a report that cites the controls.
Why a hosted audit collapses the work
Instead of running three tools, reading three formats, and stitching together a spreadsheet, a hosted audit runs CIS, CISA SCUBA, EIDSCA, ORCA, and community best practices in a single pass across Entra ID, Exchange, Teams, SharePoint, and Defender. Every finding is mapped to its control and ranked by severity, and the output is a dated PDF.
You get the coverage of all three free approaches plus the frameworks none of them cover — and a deliverable instead of raw output. See the benchmark guides for the detail, or the sample report for the format.
The short version
ORCA is your Exchange lens. EIDSCA is your identity lens. Secure Score is your headline number. None is an audit. Use them, but recognize that a real Microsoft 365 security audit covers every surface, maps to real frameworks, and produces something you can hand over.
Run an audit to get all of it in one pass.