Cloud security posture management (CSPM) can be a exhausting task. One would need to sleep one eye open to keep up with all the changes in the cloud and would still fall short. If your organization is in multi-cloud environment, then the headache is multiplied. Native tools to manage CSPM exist and you should absolutely leverage them and often their paid plans too! Microsoft Cloud environments have Secure Score and for example Identity Secure Score for Entra ID that tells you your cloud security maturity and posture. But in April 2024 a group of people consisting from couple of active MVPs and a Microsoft Identity Principal Product Manager published their community project. In this post we will look at this tool and what it can be used for. Let’s start!
Setting the goal
In this blog post we will learn what Maester is and what you can use it for. We will dip our feet briefly into the technical stuff of Maester to understand what’s under the hood and you will find links on how you can contribute to the Maester project. At last, we’ll compare Maester against other tooling: native and open-source.
What is Maester?
Strictly speaking Maester is an open-source PowerShell tool that leverages Pester, which is a testing and mocking framework for PowerShell. With Pester you can run tests and validate that your configuration (term ‘configuration’ used very loose here as Pester is very flexible) meets your expectations and requirements. Pester PowerShell module is a requirement for Maester. Other requirements are Microsoft Graph authentication module and the Maester module itself.
NOTE! If you’re using the latest v0.1 and you want to run optional CISA tests, you also need the following PowerShell modules: Az and ExchangeOnlineManagement. By default these tests are skipped if the modules are not installed.
What can you do with Maester? Like native tools in Microsoft Azure, Microsoft Defender and Microsoft Entra, you can automate your security configuration testing and monitoring. But what you most often cannot do with native tools is you cannot author your custom security target state that might differ from official recommendation. The feature that I’m most hyped about is the Conditional Access What-If tests! At the time of writing Entra portal experience offers the original What-If GUI tool, but you can also enable preview feature of What-If v2 GUI experience which is more robust. What-If tests are a good way to validate and analyze any possible gaps in our Conditional Access configuration. Our environments are under constant change, nothing is static anymore and so this type of continuous testing (that native tools do not at the moment offer) is really valuable to spot misconfigurations that might otherwise go unnoticed. Maester has capability to send overview summary report to email as well as generating interactive HTLM report file, which can be compared against each other.
Technically speaking, you can achieve all the same tests with Pester framework alone, but I don’t really see a point as you’ve got a lot of groundwork covered with Maester including the result report part that Pester does not cover. Currently Maester has over 120 built-in tests and the number in growing. Test cases at the time of writing are emphasized on Microsoft Entra, but in version 1.0 there are first test for Exchange Online too and I have a feeling that by the end of year we have many other Microsoft 365 targeted tests.
You can find documentation for both Maester and Pester at:
- Maester: https://maester.dev/
- Pester: https://pester.dev/
Maester is at version 0.1 (June 24th), so we’re at the very early stage of the development, but luckily it has gained community momentum which is crucial in these open-source projects. Expect the latest version presented in this blog to be already outdated at the time when you read this. There is a great promise and potential in this tool. Some features are on par with the native tools which are more or less expected by all of us and some features are most likely never to be supported by native tools. Without the interactive reporting, automation features and support for flexible customization out-of-the-box when Maester was released initially in April 2024, this whole project might have not gained the momentum it has.
Automation capabilities supported by Maester are currently covered by Azure Automate, Azure DevOps and GitHub. With any of these tools you can automate running of Maester tests and if you’re really handy with the PowerShell you could automate the comparison of the results.
At this point is good to mention that Maester is heavily reliant on Microsoft Graph API (graph.microsoft.com). Azure has a different API endpoint (management.azure.com). But current tests are not limited to Microsoft Graph only, now that the built-in tests also include controls from CISA’s SCuBA (Secure Cloud Business Applications Project). Running these tests will also include tests that require Az and ExchangeOnlineManagement PowerShell modules.
Security Policies as Code. What security professionals are traditionally not as used to as developers and cloud architects, is managing code. For years we’ve had a possibility to author custom Azure Policies that audit the compliance of cloud workloads and prevent unwanted misconfigurations from happening. Now we have a capability to programmatically author our own custom security policies and have those in version control. Similar to Infrastructure as Code, many seasoned system admins who transitioned to cloud specialists had to learn Infrastructure-as-Code (ARM, Terraform, Bicep, AWS CloudFormation, Pulumi). Not all security professionals have developer backgrounds, so for some this is new territorial conquest. For those who have Git and version control past, are delighted and greeting this opportunity to manage these policies as code.
Technicalities of Maester
Repositories
Maester has two public repositories in GitHub, shown in the below image. You can think of the repository called ‘maester’ having the main logic of the Maester PowerShell module and the other repository called ‘maester-tests’ listing all the built-in tests.
The logic for these repositories is simple:
- When you install Maester module with command Install-Module -Name Maester you’re using the logic from ‘maester’ repository.
- When you install Maester tests with command Install-MaesterTests .\tests you’re fetching tests from ‘maester-tests’ repository.
Installing PowerShell module(s)
As prerequisite you should install Pester and Microsoft.Graph.Authentication modules. If you already have them, update them to the latest version.
>_ Install-Module Pester -Scope CurrentUser -SkipPublisherCheck -Force>_ Install-Module Maester -Scope CurrentUser>_ Install-Module Microsoft.Graph.Authentication -Scope CurrentUser -Force
Installing tests
>_ Install-MaesterTests .\tests
First navigate to the folder where you want to install the tests and then use the above command
Installing Maester tests will create categorized folders for built-in and custom tests. If you end up authoring custom tests, place them in the ‘.\tests\Custom’ folder.
Consent
Notice that if you’re running Maester “out-of-the-box” using >_ Connect-Maester it uses Microsoft Graph Command Line Tools which, for the first time running, you have to give consent to. Once consent is given, subsequent runs won’t obviously request consent.
If you want to be able to send assessment results via email, you should use >_ Connect-Maester -SendMail. SendMail flag will add Graph permissions for email sending which requires a consent.
Running
>_ Connect-Maester>_ Invoke-Maester
Disconnect
>_ Disconnect-Maester
Disconnecting Maester will disconnect the Microsoft Graph session.
Results
Results of the assessment will be stored in the .\test-results folder. It contains three files (see the below picture). The .html file is automatically opened in browser after run.
Updating tests
Navigate to your test folder to update Maester tests, or specify path to the folder. Maester tests are pulled from the ‘maester-tests’ repository.
>_ Update-MaesterTests>_ Update-MaesterTests -Path .\tests
Frequently used commands
Maester website has comprehensive documentation of available commands. Here are the most relevant commands you should know when you start using Maester.
INSTALL AND UPDATE TESTS
Install-MaesterTests .\tests
Update-MaesterTests -Path .\tests
CONNECT AND DISCONNECT
Connect-Maester
Connect-Maester -SendMail
Disconnect-Maester
RUN MAESTER
Invoke-Maester
OTHER COMMANDS
Get-MtGraphScope [List Maester Graph scopes]
Get-MtLicenseInformation -Product EntraID [Get tenant license information]
Update-Module -Name Maester [Update Maester module]Can I contribute to the Maester project?
Yes please! Maester website has an extensive guidance for contributors. If you’re looking to develop Maester capabilities, authoring new tests or improving the documentation or you just have an idea/feature request, all contribution is welcome.
Documentation
Simple documentation changes are editable via the website and for more extensive additions or edits fork the project and make a pull request. Documentation for Maester commands are auto-generated from comment-based help in the .ps1 file.
Tests
You can contribute by writing new tests or updating existing ones. See detailed instructions here.
PowerShell module
See instructions to develop Maester PowerShell module here.
Maester vs. native tools
Here is my reflection of Maester compared to Microsoft native tools.
Microsoft Defender for Cloud
While there is definitely some overlap between the two, there are notable differences.
- Maester is not a multi-cloud tool like Microsoft Defender for Cloud (MDC)
- MDC runs mostly on top of Azure Policies which are customizable, but they only cover Azure environment
- Maester does not (yet) offer regulatory compliance built-in test
- MDC assessment is continuous, but for Maester you need to set up the automation yourself
At this point in time, I would continue using MDC as-is and start exploring what value I can add by running Maester and maybe writing my own custom tests.
Azure Advisor
Azure Advisor is a good service which has its Security pillar, but these are same recommendations as what MDC gives you, based on Microsoft Cloud Security Benchmark (MSCB). MSCB follows and encompasses controls from CIS and NIST where applicable to cloud.
- Azure Advisor only covers Azure environment
- Azure Advisor is not customizable
I don’t see Maester and Advisor competing at all. Advisor servers a different demographic: generic cloud engineers maintaining Azure environments
Azure Policy
The biggest engine behind MDC is Azure Policy. In comparison to Maester though we have one major difference: policy effects
- Azure Policies only cover Azure environment
- Azure Policies can be used to enforce configuration or deny unwanted deployments among just auditing
- Maester is “audit-only” tool
- Both are customizable and be used with version control
Azure Policy is a must have service for anyone running cloud workloads in Azure, but it currently does not reach beyond Azure. I see Maester complementing Azure Policy really well.
Maester vs. other open-source tools
Microsoft365DSC
Microsoft365DSC has very much similar characteristics to Maester, but tries to solve a bigger problem. It can be similarly used to monitor tenant configuration, but it’s also used on other things and in a larger scope.
- M365DSC used for having Microsoft 365 tenant configuration as code in version control
- Uses PowerShell Desired State Configuration (DSC) format
- M365DSC can be used for
- comparing two distinct tenant configurations
- comparing tenant configuration snapshots from different points in time
- deploying configuration
- generating configuration reports
- M365DSC is not just a security tool, it’s more
- Maester is more focused on just security and better suited for security administrators
- Maester uses Pester
Microsoft365DSC is a swiss-army knife where Maester is more of a precision tool for security pros. If you’re running Microsoft365DSC and have invested into it, there is no point switching to Maester. If you’re only interested in cloud security and nothing else then you’re up and running with Maester sooner then M365DSC.
PSRule for Azure
PSRule for Azure also has similar characteristics to Maester, but some differences can again be found.
- PSRule for Azure is based on Well-Architected Framework, is more mature with over 400 tests at the time of writing.
- PSRule for Azure has built-in controls/tests called baselines. Baselines are like packages for Rules (=tests/controls).
- PSRule for Azure only has controls for Azure environment
- PSRule for Azure allows writing own custom rules, much like Maester
- PSRule for Azure can be used to validate configuration before or after deployment using Template Analyzer (supports ARM and Bicep)
- It has native extensions to Azure DevOps, GitHub Actions and Visual Studio Code
PSRule for Azure is a great way to assess your configuration today, but I would also recommend using it to validate IaC deployment templates before deployment. It has more features than Maester, but is limited to Azure environments at the moment. Choosing between the two might come down to preferences, when it comes to monitoring Azure security. Today PSRule for Azure is more matured tool for Azure, but lacks Entra ID and other M365 capabilities. Maester will have Azure tests along the way, but it might take some time. Both have neat integrations, but Maesters html reporting and Conditional Access what-if analyzing capabilities makes it more seductive option for me.
Today you would choose PSRule for Azure when you want to assess Azure security and Maester when you want to assess Entra ID security.
Summary
All in all, Maester is very promising community project. Time will tell if the momentum is enough to keep the development of the tool rolling with frequent updates. For me the charm at the moment is the ease of use and customizability. I would not dump any of the existing and automatically operating native tools, but I would add Maester tests as an additional layer of security monitoring which can accomplish my specific customized needs. I’m hoping future releases can offer better or easier solution for reporting monitored trend of the test results as part of the current test result report. I see Maester fulfilling the needs for custom controls very well already. I recommend all security professionals working in Microsoft Entra environments to try it out. It only takes 30 minutes maximum, I promise!
Stay Secure!
