One of the genuinely good things about Microsoft 365 security is how much of the tooling is free and open. You can assess a tenant against real, well-designed baselines without buying anything. The trick is knowing what each tool covers — and what it does not.
This post is a roundup of the open-source Microsoft 365 security tools and baselines worth knowing, what each one is for, and how they fit together.
ORCA — Exchange configuration
ORCA (Office 365 Recommended Configuration Analyzer) is Microsoft's free analyzer for Exchange Online and tenant mail configuration. It compares your settings to Microsoft's recommended baseline and reports deviations.
- Use it for: Exchange and mail-flow configuration, anti-phishing defaults, catching mail misconfiguration.
- Skip it for: identity. ORCA is not an Entra ID assessment.
- Read more: What ORCA is and what it doesn't check.
EIDSCA — Entra ID configuration
EIDSCA (Entra ID Security Config Analyzer) is an open-source baseline for Entra ID, published with the Entra ID Attack and Defense Playbook. It checks identity configuration — authentication methods, Conditional Access, consent, guest defaults, privileged roles — and returns findings mapped to control IDs.
- Use it for: the identity layer, which is where most attacks land.
- Skip it for: non-identity workloads.
- Read more: EIDSCA explained.
CISA SCUBA — the government baseline
SCUBA (Secure Cloud Business Applications) is CISA's project publishing secure configuration baselines for Microsoft 365 and other cloud services, with assessment tooling. It is backed by a binding directive for federal agencies and is the most rigorous public baseline available.
- Use it for: government/compliance-driven assessments, cross-workload coverage.
- Skip it for: nothing, if you can spare the setup — but expect PowerShell and permissions work.
- Read more: CISA SCUBA for Microsoft 365.
CIS Microsoft 365 Foundations Benchmark
The CIS Microsoft 365 Foundations Benchmark is a versioned, prescriptive set of configuration controls across identity, Exchange, SharePoint, Teams, and audit — the closest thing to an industry-standard checklist.
- Use it for: broad coverage and auditor-recognizable control IDs.
- Skip it for: nothing, but budget time — it is long.
Microsoft Secure Score
Secure Score is not open source, but it is free and built into your tenant. It is a useful headline metric and source of recommendations — but it is a score, not an audit. See why Secure Score is not a security audit.
How they fit together
These tools are complementary, not competing:
| Tool | Surface | Output | | --- | --- | --- | | ORCA | Exchange / mail | Technical report | | EIDSCA | Entra ID | Findings by control | | CISA SCUBA | M365 workloads | Policy results | | CIS M365 | Broad configuration | Control pass/fail | | Secure Score | Aggregated posture | A number |
Run them by hand and you get five formats and a spreadsheet. Run them together, on a schedule, and you get a real audit: coverage across every surface, findings mapped to controls, and a report you can hand over.
That is the model behind M365Audit — it runs CIS, CISA SCUBA, EIDSCA, ORCA, and community best practices in a single pass across Entra ID, Exchange, Teams, SharePoint, and Defender, and produces a severity-ranked PDF. See the benchmark guides for the coverage detail.
The short version
The open-source Microsoft 365 security stack is strong: ORCA for Exchange, EIDSCA for identity, CISA SCUBA and CIS for broad, auditable coverage, plus Secure Score as a pulse. The gap is not the tools — it is the operational work of running them, reconciling the output, and turning it into evidence. Close that gap, and a genuinely rigorous audit is within reach of any team.
Run an audit to get all five benchmarks across every surface, in one report.